[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:10 dist-test-slave-z0sp krb5kdc[12826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:10 dist-test-slave-z0sp krb5kdc[12826](info): set up 2 sockets
Jun 20 08:09:10 dist-test-slave-z0sp krb5kdc[12826](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 08:09:11.548765 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.864s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:11 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:11.668354 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:41189
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:41189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:11.748530 12839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:11.748773 12839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:11.748807 12839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:11.749924 12839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:11.749992 12839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:11.750027 12839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:11.750053 12839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:11.750085 12839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:11.751544 12839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:41189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:41189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:11.751808 12839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:11.752035 12839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:11.758016 12847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:11.759423 12844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:11.759534 12845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:11.760077 12839 server_base.cc:1034] running on GCE node
I20260620 08:09:11.760340 12839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:11.760668 12839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:11.761907 12839 hybrid_clock.cc:648] HybridClock initialized: now 1781942951761881 us; error 33 us; skew 500 ppm
Jun 20 08:09:11 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942951, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:11.765288 12839 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:11.765751 12839 webserver.cc:460] Webserver started at http://127.12.132.254:33271/ using document root <none> and password file <none>
I20260620 08:09:11.765935 12839 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:11.765975 12839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:11.766078 12839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:11.767221 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "a547ddcafbdd458c9d108a0851472fd3"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "8cf448d4b2a41a81bef7e2ff54d2d286"
server_key_iv: "7c3a0033ab4c3631e9ef77e3018fc03b"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.767802 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "a547ddcafbdd458c9d108a0851472fd3"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "8cf448d4b2a41a81bef7e2ff54d2d286"
server_key_iv: "7c3a0033ab4c3631e9ef77e3018fc03b"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.769246 12839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:11.770049 12854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:11.770278 12839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:11.770394 12839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "a547ddcafbdd458c9d108a0851472fd3"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "8cf448d4b2a41a81bef7e2ff54d2d286"
server_key_iv: "7c3a0033ab4c3631e9ef77e3018fc03b"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.770457 12839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:11.785002 12839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:11.787286 12839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:11.787429 12839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:11.791742 12839 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:41189
I20260620 08:09:11.791796 12906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:41189 every 8 connection(s)
I20260620 08:09:11.792145 12839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:11.793105 12907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:11.793332 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 12839
I20260620 08:09:11.793417 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:11.794065 12819 external_mini_cluster.cc:1428] Setting key a6de62fe988e30ab94ddc8d57ef8f8ac
I20260620 08:09:11.796200 12907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3: Bootstrap starting.
I20260620 08:09:11.796945 12907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:11.797259 12907 log.cc:824] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:11.798039 12907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3: No bootstrap required, opened a new log
I20260620 08:09:11.800845 12907 raft_consensus.cc:348] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } }
I20260620 08:09:11.801007 12907 raft_consensus.cc:374] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:11.801031 12907 raft_consensus.cc:729] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a547ddcafbdd458c9d108a0851472fd3, State: Initialized, Role: FOLLOWER
I20260620 08:09:11.801178 12907 consensus_queue.cc:260] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } }
I20260620 08:09:11.801239 12907 raft_consensus.cc:388] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:11.801260 12907 raft_consensus.cc:482] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:11.801329 12907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:11.801836 12907 raft_consensus.cc:504] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } }
I20260620 08:09:11.801918 12907 leader_election.cc:302] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a547ddcafbdd458c9d108a0851472fd3; no voters:
I20260620 08:09:11.802153 12907 leader_election.cc:288] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:11.802176 12912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:11.802284 12912 raft_consensus.cc:686] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [term 1 LEADER]: Becoming Leader. State: Replica: a547ddcafbdd458c9d108a0851472fd3, State: Running, Role: LEADER
I20260620 08:09:11.802361 12912 consensus_queue.cc:237] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } }
I20260620 08:09:11.802408 12907 sys_catalog.cc:564] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:11.802670 12912 sys_catalog.cc:455] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a547ddcafbdd458c9d108a0851472fd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } } }
I20260620 08:09:11.802796 12912 sys_catalog.cc:458] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:11.802982 12912 sys_catalog.cc:455] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a547ddcafbdd458c9d108a0851472fd3. Latest consensus state: current_term: 1 leader_uuid: "a547ddcafbdd458c9d108a0851472fd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a547ddcafbdd458c9d108a0851472fd3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41189 } } }
I20260620 08:09:11.803100 12912 sys_catalog.cc:458] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:11.803442 12918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:11.804019 12918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:11.805495 12918 catalog_manager.cc:1347] Generated new cluster ID: 489338b994994140a46e2a7c2cf0502f
I20260620 08:09:11.805752 12918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:11 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:11.813661 12918 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:11.814026 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:11.794952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.12.132.254:41189)
0620 08:09:11.795115 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:11.795123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:11.795563 (+ 440us) server_negotiation.cc:407] Connection header received
0620 08:09:11.798339 (+ 2776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:11.798346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:11.798392 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:11.798503 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:11.800675 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:11.801913 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:11.803411 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:11.804051 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:11.804156 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:11.808850 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:11.808858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:11.808858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:11.808878 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:11.811503 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:11.812074 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:11.812078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:11.812078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:11.812380 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:11.812755 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:11.812757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:11.812757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:11.813085 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:11.813263 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:11.813563 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 08:09:11.813707 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260620 08:09:11.814426 12918 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:11.821777 12918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a547ddcafbdd458c9d108a0851472fd3: Generated new TSK 0
I20260620 08:09:11.821944 12918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:11.851919 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:11.937927 12933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:11.938126 12933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:11.938149 12933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:11.939272 12933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:11.939314 12933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:11.939342 12933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:11.940802 12933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:11.941036 12933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:11.941268 12933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:11.946151 12938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:11.946151 12939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:11.946169 12933 server_base.cc:1034] running on GCE node
W20260620 08:09:11.946375 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:11.946597 12933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:11.946902 12933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:11.948160 12933 hybrid_clock.cc:648] HybridClock initialized: now 1781942951948133 us; error 28 us; skew 500 ppm
Jun 20 08:09:11 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942951, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:11.951377 12933 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:11.952015 12933 webserver.cc:460] Webserver started at http://127.12.132.193:41229/ using document root <none> and password file <none>
I20260620 08:09:11.952306 12933 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:11.952355 12933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:11.952576 12933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:11.953830 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "d65bcf803fcc4bf3a36c62b521461462"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "d0e890d1eaa778bb698ab2d594fb1220"
server_key_iv: "d0ffec6f423883e88569f8e0c846ebb2"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.954236 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "d65bcf803fcc4bf3a36c62b521461462"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "d0e890d1eaa778bb698ab2d594fb1220"
server_key_iv: "d0ffec6f423883e88569f8e0c846ebb2"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.955881 12933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:11.956830 12948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:11.957002 12933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:11.957115 12933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "d65bcf803fcc4bf3a36c62b521461462"
format_stamp: "Formatted at 2026-06-20 08:09:11 on dist-test-slave-z0sp"
server_key: "d0e890d1eaa778bb698ab2d594fb1220"
server_key_iv: "d0ffec6f423883e88569f8e0c846ebb2"
server_key_version: "encryptionkey@0"
I20260620 08:09:11.957203 12933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:11.985711 12933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:11.988197 12933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:11.988345 12933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:11.988626 12933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:11.988974 12933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:11.989037 12933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:11.989079 12933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:11.989107 12933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:11.995044 12933 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:44473
I20260620 08:09:11.995117 13061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:44473 every 8 connection(s)
I20260620 08:09:11.995441 12933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:11.997560 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 12933
I20260620 08:09:11.997817 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:11.998046 12819 external_mini_cluster.cc:1428] Setting key fac2bafbc08d529143a098ffbed1380a
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942951, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
I20260620 08:09:12.007423 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:11.996267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36781 (local address 127.12.132.254:41189)
0620 08:09:11.996357 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:11.996359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:11.996563 (+ 204us) server_negotiation.cc:407] Connection header received
0620 08:09:11.997092 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:11.997094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:11.997106 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:11.997176 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:11.999072 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:11.999602 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.001098 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.001441 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.001532 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.004510 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:12.004512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:12.004512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:12.004520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:12.005959 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.006384 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.006385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.006386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.006444 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.006729 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.006730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.006730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.006843 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:12.006918 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:12.007230 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 08:09:12.007326 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 08:09:12.007583 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:11.996172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41189 (local address 127.12.132.193:36781)
0620 08:09:11.996356 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:11.996398 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:11.996954 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:11.997404 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:11.997411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:11.998504 (+ 1093us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:11.998954 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:11.998964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:11.999702 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:11.999703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.000992 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:12.000994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.001554 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.001554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.001811 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.002512 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:12.002523 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:12.004391 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:12.006070 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.006076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.006079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.006306 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.006534 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.006534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.006535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.006651 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.006910 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:12.006916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:12.007078 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 08:09:12.007184 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 08:09:12.007347 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260620 08:09:12.008105 13062 heartbeater.cc:347] Connected to a master server at 127.12.132.254:41189
I20260620 08:09:12.008191 13062 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:12.008378 13062 heartbeater.cc:511] Master 127.12.132.254:41189 requested a full tablet report, sending...
I20260620 08:09:12.008960 12871 ts_manager.cc:194] Registered new tserver with Master: d65bcf803fcc4bf3a36c62b521461462 (127.12.132.193:44473)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
I20260620 08:09:12.009647 12871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:36781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:12.023669 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:12.106740 13067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:12.107005 13067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:12.107043 13067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:12.108647 13067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:12.108697 13067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:12.108736 13067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:12.110625 13067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:12.110878 13067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:12.111079 13067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:12.116222 13072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:12.116204 13075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:12.116185 13073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:12.117305 13067 server_base.cc:1034] running on GCE node
I20260620 08:09:12.117573 13067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:12.117910 13067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:12.119088 13067 hybrid_clock.cc:648] HybridClock initialized: now 1781942952119058 us; error 52 us; skew 500 ppm
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:12.122340 13067 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:12.122854 13067 webserver.cc:460] Webserver started at http://127.12.132.194:43129/ using document root <none> and password file <none>
I20260620 08:09:12.123090 13067 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:12.123158 13067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:12.123310 13067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:12.124414 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "06a1ecfb0dd84c77b202be9af6611cdc"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "ae1d1bb65e1aee962a2d7f9f8f9e62ad"
server_key_iv: "7f8f05819f440961a2227d6eb1e4715a"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.124873 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "06a1ecfb0dd84c77b202be9af6611cdc"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "ae1d1bb65e1aee962a2d7f9f8f9e62ad"
server_key_iv: "7f8f05819f440961a2227d6eb1e4715a"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.126334 13067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:12.127530 13082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.127817 13067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:12.127951 13067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "06a1ecfb0dd84c77b202be9af6611cdc"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "ae1d1bb65e1aee962a2d7f9f8f9e62ad"
server_key_iv: "7f8f05819f440961a2227d6eb1e4715a"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.128048 13067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:12.153381 13067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:12.155900 13067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:12.156162 13067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:12.156400 13067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:12.156745 13067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:12.156802 13067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.156844 13067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:12.156872 13067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.163638 13067 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:43363
I20260620 08:09:12.163679 13195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:43363 every 8 connection(s)
I20260620 08:09:12.163981 13067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:12.169059 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13067
I20260620 08:09:12.169159 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:12.169342 12819 external_mini_cluster.cc:1428] Setting key 8437319c7430c4bc000755b5a5b44887
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:12.176298 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.164715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:58847 (local address 127.12.132.254:41189)
0620 08:09:12.164761 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:12.164763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:12.165413 (+ 650us) server_negotiation.cc:407] Connection header received
0620 08:09:12.166075 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:12.166077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:12.166093 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:12.166149 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:12.167354 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.167963 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.169825 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.170108 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.170187 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.173204 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:12.173206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:12.173207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:12.173216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:12.174815 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.175298 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.175299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.175300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.175370 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.175630 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.175630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.175631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.175754 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:12.175831 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:12.176116 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 08:09:12.176201 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:12.176553 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.164707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41189 (local address 127.12.132.194:58847)
0620 08:09:12.165176 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:12.165245 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:12.165938 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:12.166264 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:12.166270 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:12.166985 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:12.167248 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:12.167253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.168070 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.168071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.169682 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:12.169685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.170232 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.170233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.170407 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.171146 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:12.171156 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:12.173061 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:12.174962 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.174969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.174972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.175212 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.175439 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.175440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.175440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.175554 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.175822 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:12.175828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:12.175983 (+ 155us) client_negotiation.cc:769] Sending connection context
0620 08:09:12.176098 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 08:09:12.176267 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260620 08:09:12.177026 13196 heartbeater.cc:347] Connected to a master server at 127.12.132.254:41189
I20260620 08:09:12.177136 13196 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:12.177333 13196 heartbeater.cc:511] Master 127.12.132.254:41189 requested a full tablet report, sending...
I20260620 08:09:12.177700 12871 ts_manager.cc:194] Registered new tserver with Master: 06a1ecfb0dd84c77b202be9af6611cdc (127.12.132.194:43363)
I20260620 08:09:12.178095 12871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:58847
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:12.196007 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:12.278532 13201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:12.278721 13201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:12.278744 13201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:12.279807 13201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:12.279842 13201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:12.279868 13201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:12.281308 13201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:41189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:12.281490 13201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:12.281644 13201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:12.286365 13207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:12.286384 13206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:12.286551 13201 server_base.cc:1034] running on GCE node
W20260620 08:09:12.286478 13209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:12.286787 13201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:12.287010 13201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:12.288163 13201 hybrid_clock.cc:648] HybridClock initialized: now 1781942952288133 us; error 45 us; skew 500 ppm
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:12.290665 13201 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:12.291134 13201 webserver.cc:460] Webserver started at http://127.12.132.195:42051/ using document root <none> and password file <none>
I20260620 08:09:12.291359 13201 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:12.291426 13201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:12.291559 13201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:12.292744 13201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "eb5c722b5f6e42768fc662af750b7ee1"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "8b130a767d8b595be26652247f4a24a1"
server_key_iv: "923c446c9f433dd81db0a49bd56411b3"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.293097 13201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "eb5c722b5f6e42768fc662af750b7ee1"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "8b130a767d8b595be26652247f4a24a1"
server_key_iv: "923c446c9f433dd81db0a49bd56411b3"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.294389 13201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:12.295284 13216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.295581 13201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:12.295680 13201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "eb5c722b5f6e42768fc662af750b7ee1"
format_stamp: "Formatted at 2026-06-20 08:09:12 on dist-test-slave-z0sp"
server_key: "8b130a767d8b595be26652247f4a24a1"
server_key_iv: "923c446c9f433dd81db0a49bd56411b3"
server_key_version: "encryptionkey@0"
I20260620 08:09:12.295761 13201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:12.307176 13201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:12.309583 13201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:12.309756 13201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:12.309970 13201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:12.310269 13201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:12.310324 13201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.310365 13201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:12.310393 13201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:12.317054 13201 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:41783
I20260620 08:09:12.317092 13329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:41783 every 8 connection(s)
I20260620 08:09:12.317435 13201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:12.320771 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13201
I20260620 08:09:12.320842 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:12.321009 12819 external_mini_cluster.cc:1428] Setting key a139205c57a17371c84c780e55600e8b
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:12.327307 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.318102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:40509 (local address 127.12.132.254:41189)
0620 08:09:12.318175 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:12.318177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:12.318325 (+ 148us) server_negotiation.cc:407] Connection header received
0620 08:09:12.318768 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:12.318769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:12.318779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:12.318830 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:12.319696 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.320204 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.321782 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.322096 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.322169 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.324227 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:12.324228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:12.324229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:12.324235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:12.325684 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.326061 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.326061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.326062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.326111 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.326324 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.326324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.326325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.326415 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:12.326650 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:12.327090 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 08:09:12.327154 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:09:12.327556 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.318060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41189 (local address 127.12.132.195:40509)
0620 08:09:12.318213 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:12.318246 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:12.318691 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:12.318881 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:12.318884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:12.319429 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:12.319615 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:12.319619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.320453 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.320454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.321695 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:12.321698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.322180 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.322181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:12.322283 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.322687 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:12.322694 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:12.324143 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:12.325758 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.325763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.325765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.325985 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.326175 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:12.326176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:12.326176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:12.326271 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:12.326694 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:12.326706 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:12.326896 (+ 190us) client_negotiation.cc:769] Sending connection context
0620 08:09:12.327031 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 08:09:12.327267 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260620 08:09:12.328241 13330 heartbeater.cc:347] Connected to a master server at 127.12.132.254:41189
I20260620 08:09:12.328339 13330 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:12.328568 13330 heartbeater.cc:511] Master 127.12.132.254:41189 requested a full tablet report, sending...
I20260620 08:09:12.328938 12871 ts_manager.cc:194] Registered new tserver with Master: eb5c722b5f6e42768fc662af750b7ee1 (127.12.132.195:41783)
I20260620 08:09:12.329272 12871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:40509
I20260620 08:09:12.333559 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:12.357851 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.350149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.12.132.193:44473)
0620 08:09:12.350354 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:12.350358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:12.350389 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:09:12.350414 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:12.350416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:12.350429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:12.350485 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:12.351104 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.351898 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.352684 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.352989 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.353061 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.355139 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:12.355146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:12.355146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:12.355153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:12.356821 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.357145 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.357148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.357148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.357211 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.357377 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.357378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.357378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.357496 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:12.357596 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:12.357662 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 08:09:12.357721 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
W20260620 08:09:12.358243 12976 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51424
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[12826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:12.382725 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:12.374832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.12.132.193:44473)
0620 08:09:12.374921 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:12.374923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:12.375000 (+ 77us) server_negotiation.cc:407] Connection header received
0620 08:09:12.375061 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:12.375062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:12.375074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:12.375132 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:12.375674 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.376166 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.377100 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:12.377300 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:12.377360 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:12.380029 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:12.380030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:12.380030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:12.380038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:12.381520 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.381851 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.381851 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.381852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.381899 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:12.382087 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:12.382088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:12.382088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:12.382203 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:12.382313 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:12.382541 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 08:09:12.382651 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 08:09:12.383476 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 12933
I20260620 08:09:12.389114 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13067
I20260620 08:09:12.395177 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13201
I20260620 08:09:12.399618 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 12839
2026-06-20T08:09:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2743 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[13344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[13344](info): set up 2 sockets
Jun 20 08:09:12 dist-test-slave-z0sp krb5kdc[13344](info): commencing operation
krb5kdc: starting...
W20260620 08:09:13.460624 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:13 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:13.583545 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45005
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:45005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:13.665930 13357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:13.666134 13357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:13.666157 13357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:13.667299 13357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:13.667346 13357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:13.667361 13357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:13.667382 13357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:13.667394 13357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:13.669394 13357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:45005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:13.669646 13357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:13.669869 13357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:13.675132 13365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:13.675098 13357 server_base.cc:1034] running on GCE node
W20260620 08:09:13.675120 13362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:13.675123 13363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:13.675519 13357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:13.675804 13357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:13.676975 13357 hybrid_clock.cc:648] HybridClock initialized: now 1781942953676954 us; error 36 us; skew 500 ppm
Jun 20 08:09:13 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:13.679680 13357 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:13.680190 13357 webserver.cc:460] Webserver started at http://127.12.132.254:41787/ using document root <none> and password file <none>
I20260620 08:09:13.680410 13357 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:13.680476 13357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:13.680655 13357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:13.681679 13357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "26e84557c3f84ecc9569089accd10bd5"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "1467b2bdac80091399daa5dfa84654dd"
server_key_iv: "61763d777635ec6774756e89739ee71c"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.682042 13357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "26e84557c3f84ecc9569089accd10bd5"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "1467b2bdac80091399daa5dfa84654dd"
server_key_iv: "61763d777635ec6774756e89739ee71c"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.683425 13357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:09:13.684219 13372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:13.684408 13357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:13.684525 13357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "26e84557c3f84ecc9569089accd10bd5"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "1467b2bdac80091399daa5dfa84654dd"
server_key_iv: "61763d777635ec6774756e89739ee71c"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.684609 13357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:13.700119 13357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:13.702385 13357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:13.702543 13357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:13.706722 13357 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:45005
I20260620 08:09:13.706768 13424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:45005 every 8 connection(s)
I20260620 08:09:13.707093 13357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:13.707898 13425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:13.709021 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13357
I20260620 08:09:13.709129 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:13.709308 12819 external_mini_cluster.cc:1428] Setting key 3e4d989786aa2339b3f08ff5826c7ef7
I20260620 08:09:13.710645 13425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5: Bootstrap starting.
I20260620 08:09:13.711268 13425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:13.711602 13425 log.cc:824] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:13.712267 13425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5: No bootstrap required, opened a new log
I20260620 08:09:13.714076 13425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } }
I20260620 08:09:13.714233 13425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:13.714278 13425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e84557c3f84ecc9569089accd10bd5, State: Initialized, Role: FOLLOWER
I20260620 08:09:13.714401 13425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } }
I20260620 08:09:13.714494 13425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:13.714545 13425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:13.714593 13425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:13.715102 13425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } }
I20260620 08:09:13.715219 13425 leader_election.cc:302] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26e84557c3f84ecc9569089accd10bd5; no voters:
I20260620 08:09:13.715389 13425 leader_election.cc:288] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:13.715448 13430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:13.715593 13430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 26e84557c3f84ecc9569089accd10bd5, State: Running, Role: LEADER
I20260620 08:09:13.715668 13425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:13.715682 13430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } }
Jun 20 08:09:13 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:13.716007 13430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26e84557c3f84ecc9569089accd10bd5. Latest consensus state: current_term: 1 leader_uuid: "26e84557c3f84ecc9569089accd10bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } } }
I20260620 08:09:13.716014 13432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26e84557c3f84ecc9569089accd10bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26e84557c3f84ecc9569089accd10bd5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45005 } } }
I20260620 08:09:13.716094 13430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:13.716126 13432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:13.716324 13437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:13.717093 13437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:13.718588 13437 catalog_manager.cc:1347] Generated new cluster ID: 37c9c04229a54f7ba200c0f2719ef7b5
I20260620 08:09:13.718637 13437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:13.720316 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:13.709830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.12.132.254:45005)
0620 08:09:13.710016 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:13.710031 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:13.710064 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:09:13.710593 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:13.710601 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:13.710669 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:13.710829 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:13.711493 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.712484 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.713272 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.713763 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.713917 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:13.716637 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:13.716641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:13.716641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:13.716652 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:13.718745 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:13.719125 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:13.719128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:13.719128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:13.719241 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:13.719438 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:13.719439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:13.719440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:13.719736 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:13.719908 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:13.719979 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 08:09:13.720112 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260620 08:09:13.734316 13437 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:13.734942 13437 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:13.743044 13437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26e84557c3f84ecc9569089accd10bd5: Generated new TSK 0
I20260620 08:09:13.743242 13437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:13.779785 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:13.867329 13451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:13.867558 13451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:13.867589 13451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:13.868886 13451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:13.868938 13451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:13.868966 13451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:13.870517 13451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:13.870781 13451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:13.871052 13451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:13.877323 13457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:13.877353 13459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:13.877324 13456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:13.877767 13451 server_base.cc:1034] running on GCE node
I20260620 08:09:13.878034 13451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:13.878350 13451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:13.879552 13451 hybrid_clock.cc:648] HybridClock initialized: now 1781942953879570 us; error 62 us; skew 500 ppm
Jun 20 08:09:13 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:13.882382 13451 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:13.882812 13451 webserver.cc:460] Webserver started at http://127.12.132.193:41403/ using document root <none> and password file <none>
I20260620 08:09:13.883050 13451 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:13.883101 13451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:13.883236 13451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:13.884400 13451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "2448a6e6b11fabcf4ecc0b204cbbc287"
server_key_iv: "ce3b22c0899ab97fbf9b39e30102e40d"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.884840 13451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "2448a6e6b11fabcf4ecc0b204cbbc287"
server_key_iv: "ce3b22c0899ab97fbf9b39e30102e40d"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.886605 13451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:13.887528 13466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:13.887737 13451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:13.887852 13451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0"
format_stamp: "Formatted at 2026-06-20 08:09:13 on dist-test-slave-z0sp"
server_key: "2448a6e6b11fabcf4ecc0b204cbbc287"
server_key_iv: "ce3b22c0899ab97fbf9b39e30102e40d"
server_key_version: "encryptionkey@0"
I20260620 08:09:13.887931 13451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:13.910928 13451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:13.913239 13451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:13.913404 13451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:13.913645 13451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:13.913951 13451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:13.914002 13451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:13.914044 13451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:13.914072 13451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:13.920084 13451 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:33173
I20260620 08:09:13.920131 13581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:33173 every 8 connection(s)
I20260620 08:09:13.920447 13451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:13.926033 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13451
I20260620 08:09:13.926116 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:13.926383 12819 external_mini_cluster.cc:1428] Setting key 0e628ccc9b3581e564e6210a6691e8ad
Jun 20 08:09:13 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:13.930194 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:13.921176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:46365 (local address 127.12.132.254:45005)
0620 08:09:13.921223 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:13.921224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:13.921425 (+ 201us) server_negotiation.cc:407] Connection header received
0620 08:09:13.922377 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:13.922378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:13.922391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:13.922456 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:13.923059 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.923559 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.924605 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.924804 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.924884 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:13.927108 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:13.927109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:13.927109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:13.927115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:13.928918 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:13.929295 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:13.929296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:13.929296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:13.929361 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:13.929591 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:13.929592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:13.929592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:13.929707 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:13.929761 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:13.930006 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 08:09:13.930103 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:13.930371 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:13.921172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45005 (local address 127.12.132.193:46365)
0620 08:09:13.921304 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:13.921337 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:13.922267 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:13.922537 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:13.922540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:13.922780 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:13.922960 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:13.922963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.923654 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.923655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:13.924456 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:13.924458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:13.924897 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:13.924898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:13.925027 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:13.925484 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:13.925492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:13.927019 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:13.929000 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:13.929006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:13.929007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:13.929218 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:13.929428 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:13.929429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:13.929429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:13.929527 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:13.929763 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:13.929766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:13.929872 (+ 106us) client_negotiation.cc:769] Sending connection context
0620 08:09:13.929981 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 08:09:13.930131 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260620 08:09:13.930998 13582 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45005
I20260620 08:09:13.931110 13582 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:13.931282 13582 heartbeater.cc:511] Master 127.12.132.254:45005 requested a full tablet report, sending...
I20260620 08:09:13.931833 13389 ts_manager.cc:194] Registered new tserver with Master: 8e12ba5f8f64447cab03d0e3d07c4bc0 (127.12.132.193:33173)
I20260620 08:09:13.932814 13389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:46365
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:13.955111 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:14.037467 13587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:14.037667 13587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:14.037691 13587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:14.038796 13587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:14.038832 13587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:14.038857 13587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:14.040318 13587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:14.040561 13587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:14.040743 13587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:14.045782 13595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:14.045835 13593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:14.045781 13592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:14.045974 13587 server_base.cc:1034] running on GCE node
I20260620 08:09:14.046165 13587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:14.046357 13587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:14.047515 13587 hybrid_clock.cc:648] HybridClock initialized: now 1781942954047501 us; error 29 us; skew 500 ppm
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:14.050001 13587 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:14.050447 13587 webserver.cc:460] Webserver started at http://127.12.132.194:41815/ using document root <none> and password file <none>
I20260620 08:09:14.050661 13587 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:14.050706 13587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:14.050856 13587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:14.051847 13587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "cd55de48bb4e4af3af993c6e7ff967bf"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "8994d16cc25bc5eb8e8bc32556f55b66"
server_key_iv: "d9acc551b6bf94cce601341c6f806a27"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.052191 13587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "cd55de48bb4e4af3af993c6e7ff967bf"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "8994d16cc25bc5eb8e8bc32556f55b66"
server_key_iv: "d9acc551b6bf94cce601341c6f806a27"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.053536 13587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:14.054292 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.054634 13587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:14.054705 13587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "cd55de48bb4e4af3af993c6e7ff967bf"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "8994d16cc25bc5eb8e8bc32556f55b66"
server_key_iv: "d9acc551b6bf94cce601341c6f806a27"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.054790 13587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:14.075263 13587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:14.077566 13587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:14.077725 13587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:14.077984 13587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:14.078284 13587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:14.078317 13587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.078338 13587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:14.078353 13587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.084200 13587 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:41437
I20260620 08:09:14.084255 13716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:41437 every 8 connection(s)
I20260620 08:09:14.084617 13587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:14.090250 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13587
I20260620 08:09:14.090351 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:14.090540 12819 external_mini_cluster.cc:1428] Setting key a3befb46e871efc1a4a1e90f7cdf714c
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:14.096850 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.085304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40571 (local address 127.12.132.254:45005)
0620 08:09:14.085347 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.085349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.085527 (+ 178us) server_negotiation.cc:407] Connection header received
0620 08:09:14.086635 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.086638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.086737 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.086866 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.087985 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.088840 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.089914 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.090143 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.090202 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.092858 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.092860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.092861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.092873 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.094804 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.095235 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.095237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.095237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.095309 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.095603 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.095604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.095605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.096099 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.096217 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.096557 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.096714 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:14.096975 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.085265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45005 (local address 127.12.132.194:40571)
0620 08:09:14.085414 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.085454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.086425 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.087034 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.087039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.087428 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:14.087865 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.087873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.088969 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.088971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.089816 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.089819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.090233 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.090234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.090397 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.091045 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:14.091059 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:14.092736 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:14.094904 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:14.094910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:14.094912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:14.095145 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:14.095407 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:14.095408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:14.095409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:14.095519 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:14.096213 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:14.096218 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:14.096345 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.096448 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.096675 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
I20260620 08:09:14.097523 13717 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45005
I20260620 08:09:14.097635 13717 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:14.097834 13717 heartbeater.cc:511] Master 127.12.132.254:45005 requested a full tablet report, sending...
I20260620 08:09:14.098398 13389 ts_manager.cc:194] Registered new tserver with Master: cd55de48bb4e4af3af993c6e7ff967bf (127.12.132.194:41437)
I20260620 08:09:14.098964 13389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:40571
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:14.121515 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:14.201797 13722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:14.201997 13722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:14.202021 13722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:14.203199 13722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:14.203249 13722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:14.203276 13722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:14.204782 13722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:14.205004 13722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:14.205227 13722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:14.210040 13727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:14.210150 13730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:14.210363 13728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:14.210407 13722 server_base.cc:1034] running on GCE node
I20260620 08:09:14.210593 13722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:14.210836 13722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:14.212016 13722 hybrid_clock.cc:648] HybridClock initialized: now 1781942954212004 us; error 30 us; skew 500 ppm
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:14.214581 13722 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:14.215067 13722 webserver.cc:460] Webserver started at http://127.12.132.195:40547/ using document root <none> and password file <none>
I20260620 08:09:14.215289 13722 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:14.215358 13722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:14.215494 13722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:14.216678 13722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "54919ba344734e69bb044500ec0bd24c"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "e220648012a9a2f16ce5d81447ea6c52"
server_key_iv: "2f52628eeee98c3d477beb5cec345198"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.217052 13722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "54919ba344734e69bb044500ec0bd24c"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "e220648012a9a2f16ce5d81447ea6c52"
server_key_iv: "2f52628eeee98c3d477beb5cec345198"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.218642 13722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:14.219559 13737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.219861 13722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:14.219977 13722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "54919ba344734e69bb044500ec0bd24c"
format_stamp: "Formatted at 2026-06-20 08:09:14 on dist-test-slave-z0sp"
server_key: "e220648012a9a2f16ce5d81447ea6c52"
server_key_iv: "2f52628eeee98c3d477beb5cec345198"
server_key_version: "encryptionkey@0"
I20260620 08:09:14.220069 13722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:14.240262 13722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:14.242695 13722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:14.242859 13722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:14.243124 13722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:14.243427 13722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:14.243481 13722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.243525 13722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:14.243551 13722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:14.249619 13722 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:36105
I20260620 08:09:14.249703 13851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:36105 every 8 connection(s)
I20260620 08:09:14.250010 13722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:14.257987 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13722
I20260620 08:09:14.258069 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:14.258236 12819 external_mini_cluster.cc:1428] Setting key c80a4eaa388388db46cff23e6dc04678
I20260620 08:09:14.260133 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.250724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:38377 (local address 127.12.132.254:45005)
0620 08:09:14.250812 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.250814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.251019 (+ 205us) server_negotiation.cc:407] Connection header received
0620 08:09:14.251941 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.251943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.251953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.252017 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.252661 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.253182 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.254138 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.254326 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.254383 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.256410 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.256412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.256412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.256418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.258524 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.259054 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.259055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.259056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.259118 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.259348 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.259349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.259349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.259449 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.259504 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.259999 (+ 495us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.260065 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:09:14.260454 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.250693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45005 (local address 127.12.132.195:38377)
0620 08:09:14.250888 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.250928 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.251810 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.252069 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.252072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.252309 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:14.252554 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.252559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.253268 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.253269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.254058 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.254060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.254422 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.254423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.254537 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.254981 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:14.254989 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:14.256323 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:14.258635 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:14.258642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:14.258644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:14.258941 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:14.259192 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:14.259193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:14.259193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:14.259290 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:14.259589 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:14.259601 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:14.259812 (+ 211us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.259949 (+ 137us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.260121 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":113,"threads_started":1}
I20260620 08:09:14.261219 13852 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45005
I20260620 08:09:14.261353 13852 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:14.261534 13852 heartbeater.cc:511] Master 127.12.132.254:45005 requested a full tablet report, sending...
I20260620 08:09:14.261922 13389 ts_manager.cc:194] Registered new tserver with Master: 54919ba344734e69bb044500ec0bd24c (127.12.132.195:36105)
I20260620 08:09:14.262302 13389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:38377
I20260620 08:09:14.270179 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:14.278815 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.271957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.12.132.254:45005)
0620 08:09:14.272030 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.272032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.272149 (+ 117us) server_negotiation.cc:407] Connection header received
0620 08:09:14.272245 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.272246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.272260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.272312 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.273183 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.273762 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.274501 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.274697 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.274780 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.276206 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.276208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.276208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.276218 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.277590 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.277985 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.277986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.277986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.278058 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.278266 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.278267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.278268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.278414 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.278531 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.278641 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.278719 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 08:09:14.281527 13389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:14.283021 13389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:14.290904 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.286896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.12.132.194:41437)
0620 08:09:14.287125 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.287131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.287143 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:14.287183 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.287185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.287201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.287309 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:14.287452 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.288417 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.289742 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.290484 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.290575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.290635 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.290778 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.290828 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
I20260620 08:09:14.290899 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.286482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:41437 (local address 127.0.0.1:45224)
0620 08:09:14.286745 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.286800 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.286868 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.287317 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.287321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.287324 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:14.287393 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.287395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.288565 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.288566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.289650 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.289652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.290583 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.290584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.290663 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.290677 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.290753 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.290821 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":51,"threads_started":1}
I20260620 08:09:14.291601 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.286527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:33173 (local address 127.0.0.1:42058)
0620 08:09:14.287422 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.287436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.287527 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.287919 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.287920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.287923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:14.287971 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.287972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.289458 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.289460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.290301 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.290304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.291320 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.291321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.291414 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.291418 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.291472 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.291511 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":45,"thread_start_us":153,"threads_started":1}
I20260620 08:09:14.291815 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.286960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.12.132.193:33173)
0620 08:09:14.287228 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.287232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.287514 (+ 282us) server_negotiation.cc:407] Connection header received
0620 08:09:14.287708 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.287719 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.287740 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.287818 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:14.288074 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.289342 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.290374 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.291140 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.291211 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.291258 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.291704 (+ 446us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.291747 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":151,"threads_started":1}
I20260620 08:09:14.292019 13650 tablet_service.cc:1467] Processing CreateTablet for tablet fc80fcea4f3245ebbbdaaca61fb624b5 (DEFAULT_TABLE table=test-table [id=4fa2ec22bd54433ab4f656ffcd43bf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:14.292287 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.286515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:36105 (local address 127.0.0.1:57192)
0620 08:09:14.286971 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.286979 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.287035 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.287724 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.287725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.287728 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:14.287858 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.287860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.289446 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.289447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.290555 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.290558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.291638 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.291639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.292118 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.292123 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.292188 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.292226 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":166,"threads_started":1}
I20260620 08:09:14.292400 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc80fcea4f3245ebbbdaaca61fb624b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:14.292554 13868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.287139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57192 (local address 127.12.132.195:36105)
0620 08:09:14.287391 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.287397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.287411 (+ 14us) server_negotiation.cc:407] Connection header received
0620 08:09:14.287469 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.287471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.287492 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.287929 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:14.288001 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.289308 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.290645 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.291573 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.291641 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.291693 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.292248 (+ 555us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.292292 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260620 08:09:14.292461 13514 tablet_service.cc:1467] Processing CreateTablet for tablet fc80fcea4f3245ebbbdaaca61fb624b5 (DEFAULT_TABLE table=test-table [id=4fa2ec22bd54433ab4f656ffcd43bf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:14.292778 13514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc80fcea4f3245ebbbdaaca61fb624b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:14.295195 13869 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Bootstrap starting.
I20260620 08:09:14.295301 13870 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: Bootstrap starting.
I20260620 08:09:14.295948 13870 tablet_bootstrap.cc:654] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:14.296012 13785 tablet_service.cc:1467] Processing CreateTablet for tablet fc80fcea4f3245ebbbdaaca61fb624b5 (DEFAULT_TABLE table=test-table [id=4fa2ec22bd54433ab4f656ffcd43bf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:14.296347 13785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc80fcea4f3245ebbbdaaca61fb624b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:14.296600 13869 tablet_bootstrap.cc:654] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:14.296793 13870 log.cc:824] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:14.297195 13869 log.cc:824] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:14.297664 13870 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: No bootstrap required, opened a new log
I20260620 08:09:14.297739 13870 ts_tablet_manager.cc:1397] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:09:14.297984 13869 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: No bootstrap required, opened a new log
I20260620 08:09:14.298053 13869 ts_tablet_manager.cc:1397] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 08:09:14.299427 13869 raft_consensus.cc:348] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.299639 13869 raft_consensus.cc:374] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:14.299695 13869 raft_consensus.cc:729] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd55de48bb4e4af3af993c6e7ff967bf, State: Initialized, Role: FOLLOWER
I20260620 08:09:14.299786 13873 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: Bootstrap starting.
I20260620 08:09:14.299765 13870 raft_consensus.cc:348] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.299849 13869 consensus_queue.cc:260] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.299925 13870 raft_consensus.cc:374] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:14.299964 13870 raft_consensus.cc:729] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e12ba5f8f64447cab03d0e3d07c4bc0, State: Initialized, Role: FOLLOWER
I20260620 08:09:14.300076 13870 consensus_queue.cc:260] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.300120 13869 ts_tablet_manager.cc:1428] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:09:14.300238 13717 heartbeater.cc:503] Master 127.12.132.254:45005 was elected leader, sending a full tablet report...
I20260620 08:09:14.300359 13870 ts_tablet_manager.cc:1428] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:14.300405 13873 tablet_bootstrap.cc:654] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:14.300526 13582 heartbeater.cc:503] Master 127.12.132.254:45005 was elected leader, sending a full tablet report...
I20260620 08:09:14.301069 13873 log.cc:824] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:14.301782 13873 tablet_bootstrap.cc:492] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: No bootstrap required, opened a new log
I20260620 08:09:14.301839 13873 ts_tablet_manager.cc:1397] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:09:14.303596 13873 raft_consensus.cc:348] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.303722 13873 raft_consensus.cc:374] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:14.303743 13873 raft_consensus.cc:729] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54919ba344734e69bb044500ec0bd24c, State: Initialized, Role: FOLLOWER
I20260620 08:09:14.303853 13873 consensus_queue.cc:260] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.304102 13873 ts_tablet_manager.cc:1428] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 08:09:14.304137 13852 heartbeater.cc:503] Master 127.12.132.254:45005 was elected leader, sending a full tablet report...
I20260620 08:09:14.323274 13877 raft_consensus.cc:482] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:14.323367 13877 raft_consensus.cc:504] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.323716 13877 leader_election.cc:288] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e12ba5f8f64447cab03d0e3d07c4bc0 (127.12.132.193:33173), cd55de48bb4e4af3af993c6e7ff967bf (127.12.132.194:41437)
I20260620 08:09:14.326498 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.323851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:33173 (local address 127.12.132.195:46113)
0620 08:09:14.323922 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.323933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.323973 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.324174 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.324175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.324177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:14.324239 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.324240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.324932 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.324933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.325665 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.325667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.326234 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.326235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.326313 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.326316 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.326392 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.326429 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":4}
I20260620 08:09:14.326499 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.323963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:46113 (local address 127.12.132.193:33173)
0620 08:09:14.324013 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.324016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.324026 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:14.324068 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.324069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.324080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.324145 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:14.324312 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.324848 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.325770 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.326148 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.326236 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.326244 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.326401 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.326435 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:09:14.326599 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.324043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:50583 (local address 127.12.132.194:41437)
0620 08:09:14.324112 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.324115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.324205 (+ 90us) server_negotiation.cc:407] Connection header received
0620 08:09:14.324263 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.324264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.324273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.324326 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:14.324473 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.324947 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.325881 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.326256 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.326341 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.326349 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.326502 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.326537 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 08:09:14.326610 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.323970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:41437 (local address 127.12.132.195:50583)
0620 08:09:14.324126 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:14.324136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:14.324211 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:14.324330 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:14.324330 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:14.324332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:14.324403 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.324405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.325054 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.325055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.325790 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:14.325792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.326340 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.326340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:14.326416 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.326420 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:14.326494 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 08:09:14.326537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260620 08:09:14.326874 13534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc80fcea4f3245ebbbdaaca61fb624b5" candidate_uuid: "54919ba344734e69bb044500ec0bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" is_pre_election: true
I20260620 08:09:14.327000 13534 raft_consensus.cc:2454] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54919ba344734e69bb044500ec0bd24c in term 0.
I20260620 08:09:14.327215 13738 leader_election.cc:302] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54919ba344734e69bb044500ec0bd24c, 8e12ba5f8f64447cab03d0e3d07c4bc0; no voters:
I20260620 08:09:14.327203 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc80fcea4f3245ebbbdaaca61fb624b5" candidate_uuid: "54919ba344734e69bb044500ec0bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" is_pre_election: true
I20260620 08:09:14.327337 13671 raft_consensus.cc:2454] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54919ba344734e69bb044500ec0bd24c in term 0.
I20260620 08:09:14.327373 13877 raft_consensus.cc:2781] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:14.327457 13877 raft_consensus.cc:482] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:14.327492 13877 raft_consensus.cc:3037] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:14.328153 13877 raft_consensus.cc:504] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.328284 13877 leader_election.cc:288] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [CANDIDATE]: Term 1 election: Requested vote from peers 8e12ba5f8f64447cab03d0e3d07c4bc0 (127.12.132.193:33173), cd55de48bb4e4af3af993c6e7ff967bf (127.12.132.194:41437)
I20260620 08:09:14.328503 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc80fcea4f3245ebbbdaaca61fb624b5" candidate_uuid: "54919ba344734e69bb044500ec0bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd55de48bb4e4af3af993c6e7ff967bf"
I20260620 08:09:14.328469 13534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc80fcea4f3245ebbbdaaca61fb624b5" candidate_uuid: "54919ba344734e69bb044500ec0bd24c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0"
I20260620 08:09:14.328590 13671 raft_consensus.cc:3037] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:14.328614 13534 raft_consensus.cc:3037] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:14.329231 13534 raft_consensus.cc:2454] T fc80fcea4f3245ebbbdaaca61fb624b5 P 8e12ba5f8f64447cab03d0e3d07c4bc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54919ba344734e69bb044500ec0bd24c in term 1.
I20260620 08:09:14.329347 13671 raft_consensus.cc:2454] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54919ba344734e69bb044500ec0bd24c in term 1.
I20260620 08:09:14.329401 13738 leader_election.cc:302] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54919ba344734e69bb044500ec0bd24c, 8e12ba5f8f64447cab03d0e3d07c4bc0; no voters:
I20260620 08:09:14.329505 13877 raft_consensus.cc:2781] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:14.329649 13877 raft_consensus.cc:686] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [term 1 LEADER]: Becoming Leader. State: Replica: 54919ba344734e69bb044500ec0bd24c, State: Running, Role: LEADER
I20260620 08:09:14.329744 13877 consensus_queue.cc:237] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } }
I20260620 08:09:14.330991 13389 catalog_manager.cc:5495] T fc80fcea4f3245ebbbdaaca61fb624b5 P 54919ba344734e69bb044500ec0bd24c reported cstate change: term changed from 0 to 1, leader changed from <none> to 54919ba344734e69bb044500ec0bd24c (127.12.132.195). New cstate: current_term: 1 leader_uuid: "54919ba344734e69bb044500ec0bd24c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54919ba344734e69bb044500ec0bd24c" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e12ba5f8f64447cab03d0e3d07c4bc0" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd55de48bb4e4af3af993c6e7ff967bf" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41437 } health_report { overall_health: UNKNOWN } } }
W20260620 08:09:14.335112 13718 tablet.cc:2369] T fc80fcea4f3245ebbbdaaca61fb624b5 P cd55de48bb4e4af3af993c6e7ff967bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:14.341037 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.333453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.12.132.193:33173)
0620 08:09:14.333535 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.333537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.333631 (+ 94us) server_negotiation.cc:407] Connection header received
0620 08:09:14.333707 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.333709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.333717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.333773 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.334317 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.334763 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.335528 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.335736 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.335820 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.338107 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.338115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.338115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.338122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.339837 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.340205 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.340208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.340208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.340270 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.340465 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.340466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.340467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.340640 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.340795 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.340861 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.340938 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:14.365403 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.357024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.12.132.193:33173)
0620 08:09:14.357108 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.357109 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.357118 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:14.357275 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.357277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.357299 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.357388 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.358047 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.358650 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.359452 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.359667 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.359749 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.362429 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.362431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.362431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.362438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.363906 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.364282 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.364283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.364283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.364345 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.364684 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.364685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.364686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.364850 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.364959 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.365199 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.365309 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:14.391283 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:14.383142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.12.132.193:33173)
0620 08:09:14.383213 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:14.383215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:14.383296 (+ 81us) server_negotiation.cc:407] Connection header received
0620 08:09:14.383388 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:14.383389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:14.383399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:14.383458 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:14.383991 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.384472 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.385281 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:14.385481 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:14.385559 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:14.388344 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:14.388346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:14.388346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:14.388360 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:14.389830 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.390466 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.390467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.390468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.390565 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:14.390781 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:14.390781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:14.390782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:14.390895 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:14.391005 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:14.391105 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 08:09:14.391177 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260620 08:09:14.391772 13494 tablet_service.cc:2908] Rejecting scan request for tablet fc80fcea4f3245ebbbdaaca61fb624b5: Uninitialized: clean time has not yet been initialized
I20260620 08:09:14.393002 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13451
I20260620 08:09:14.398923 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13587
I20260620 08:09:14.403286 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13722
I20260620 08:09:14.408924 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13357
2026-06-20T08:09:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2009 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13893](info): set up 2 sockets
Jun 20 08:09:14 dist-test-slave-z0sp krb5kdc[13893](info): commencing operation
krb5kdc: starting...
W20260620 08:09:15.465086 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:15 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:15.587178 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36613
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:15.669327 13906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:15.669517 13906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:15.669539 13906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:15.670694 13906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:15.670737 13906 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 08:09:15.670750 13906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:15.670763 13906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:15.670781 13906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:15.670814 13906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:15.672268 13906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36613
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:15.672504 13906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:15.672770 13906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:09:15.677729 13906 server_base.cc:1034] running on GCE node
W20260620 08:09:15.677673 13911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:15.677719 13912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:15.677672 13914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:15.678202 13906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:15.678429 13906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:15.679563 13906 hybrid_clock.cc:648] HybridClock initialized: now 1781942955679544 us; error 43 us; skew 500 ppm
Jun 20 08:09:15 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:15.685616 13906 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:15.686192 13906 webserver.cc:460] Webserver started at http://127.12.132.254:42989/ using document root <none> and password file <none>
I20260620 08:09:15.686473 13906 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:15.686547 13906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:15.686709 13906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:15.688042 13906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "b15a8a332db348faa8937dbc0112204b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "55f34512473302060f6b1dc78d69db1f"
server_key_iv: "df2b13519faed2b882dc9a68c9144945"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.688757 13906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "b15a8a332db348faa8937dbc0112204b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "55f34512473302060f6b1dc78d69db1f"
server_key_iv: "df2b13519faed2b882dc9a68c9144945"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.690258 13906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:15.691144 13921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:15.691349 13906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:15.691440 13906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "b15a8a332db348faa8937dbc0112204b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "55f34512473302060f6b1dc78d69db1f"
server_key_iv: "df2b13519faed2b882dc9a68c9144945"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.691530 13906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:15.707489 13906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:15.709720 13906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:15.709897 13906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:15.714603 13906 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:36613
I20260620 08:09:15.714720 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:36613 every 8 connection(s)
I20260620 08:09:15.715011 13906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:15.715610 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:15.717830 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Bootstrap starting.
I20260620 08:09:15.718356 13985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:15.718577 13985 log.cc:824] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:15.719188 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: No bootstrap required, opened a new log
I20260620 08:09:15.721256 13985 raft_consensus.cc:348] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } }
I20260620 08:09:15.721391 13985 raft_consensus.cc:374] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:15.721414 13985 raft_consensus.cc:729] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a8a332db348faa8937dbc0112204b, State: Initialized, Role: FOLLOWER
I20260620 08:09:15.721544 13985 consensus_queue.cc:260] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } }
I20260620 08:09:15.721628 13985 raft_consensus.cc:388] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:15.721652 13985 raft_consensus.cc:482] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:15.721678 13985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:15.722134 13985 raft_consensus.cc:504] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } }
I20260620 08:09:15.722265 13985 leader_election.cc:302] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b15a8a332db348faa8937dbc0112204b; no voters:
I20260620 08:09:15.722371 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 13906
I20260620 08:09:15.722419 13985 leader_election.cc:288] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:15.722476 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:15.722488 13988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:15.722653 12819 external_mini_cluster.cc:1428] Setting key 7fd96f386d19282c254137eda743f135
I20260620 08:09:15.722656 13985 sys_catalog.cc:564] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:15.722694 13988 raft_consensus.cc:686] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [term 1 LEADER]: Becoming Leader. State: Replica: b15a8a332db348faa8937dbc0112204b, State: Running, Role: LEADER
I20260620 08:09:15.722780 13988 consensus_queue.cc:237] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } }
I20260620 08:09:15.723107 13989 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b15a8a332db348faa8937dbc0112204b. Latest consensus state: current_term: 1 leader_uuid: "b15a8a332db348faa8937dbc0112204b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } } }
I20260620 08:09:15.723142 13990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b15a8a332db348faa8937dbc0112204b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b15a8a332db348faa8937dbc0112204b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36613 } } }
I20260620 08:09:15.723279 13989 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:15.723362 13990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:15.724231 14002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:15.724767 14002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:15.726112 14002 catalog_manager.cc:1347] Generated new cluster ID: 8b0683cf17c643b791613f0e8edf98c1
I20260620 08:09:15.726166 14002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:15 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:15.733251 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:15.723169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.12.132.254:36613)
0620 08:09:15.723448 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:15.723457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:15.723490 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:09:15.723605 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:15.723613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:15.723670 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:15.723846 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:15.724636 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.725873 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.726694 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.727127 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.727253 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:15.729919 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:15.729930 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:15.729930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:15.729940 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:15.731473 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.731934 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.731939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.731939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.732099 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.732312 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.732313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.732313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.732644 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:15.732832 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:15.732897 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 08:09:15.733036 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":235,"thread_start_us":68,"threads_started":1}
I20260620 08:09:15.752205 14002 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:15.752799 14002 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:15.760311 14002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Generated new TSK 0
I20260620 08:09:15.760470 14002 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:15.788913 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36613
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:15.874053 14011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:15.874274 14011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:15.874298 14011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:15.875537 14011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:15.875581 14011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:15.875602 14011 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:15.875619 14011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:15.877502 14011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:36613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:15.877749 14011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:15.878018 14011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:15.883900 14019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:15.883826 14016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:15.883833 14017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:15.884085 14011 server_base.cc:1034] running on GCE node
I20260620 08:09:15.884279 14011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:15.884470 14011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:15.885669 14011 hybrid_clock.cc:648] HybridClock initialized: now 1781942955885658 us; error 26 us; skew 500 ppm
Jun 20 08:09:15 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:15.888401 14011 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:15.888860 14011 webserver.cc:460] Webserver started at http://127.12.132.193:46029/ using document root <none> and password file <none>
I20260620 08:09:15.889086 14011 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:15.889129 14011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:15.889295 14011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:15.890331 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "ea8f1d19331044b8830b10381637ed5b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "6953c205a9594620910402aa0e1d72cb"
server_key_iv: "1dd63c4d198442d4dea4246ba8a58220"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.890677 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "ea8f1d19331044b8830b10381637ed5b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "6953c205a9594620910402aa0e1d72cb"
server_key_iv: "1dd63c4d198442d4dea4246ba8a58220"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.892060 14011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:15.893067 14026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:15.893229 14011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:15.893358 14011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "ea8f1d19331044b8830b10381637ed5b"
format_stamp: "Formatted at 2026-06-20 08:09:15 on dist-test-slave-z0sp"
server_key: "6953c205a9594620910402aa0e1d72cb"
server_key_iv: "1dd63c4d198442d4dea4246ba8a58220"
server_key_version: "encryptionkey@0"
I20260620 08:09:15.893433 14011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:15.928380 14011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:15.930510 14011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:15.930622 14011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:15.931108 14011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:15.931145 14011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:15.931169 14011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:15.931319 14011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:15 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:15.938555 14011 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35515
I20260620 08:09:15.939224 14011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:15.939278 14142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35515 every 8 connection(s)
I20260620 08:09:15.942132 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:15.931640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:38367 (local address 127.12.132.254:36613)
0620 08:09:15.931711 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:15.931714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:15.931898 (+ 184us) server_negotiation.cc:407] Connection header received
0620 08:09:15.932552 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:15.932554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:15.932571 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:15.932669 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:15.933482 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.934315 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.935884 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.936218 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.936307 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:15.939240 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:15.939241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:15.939242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:15.939249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:15.940883 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.941349 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.941350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.941351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.941408 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.941643 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.941643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.941644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.941744 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:15.941795 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:15.941994 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 08:09:15.942071 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 08:09:15.942320 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:15.931624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.193:38367)
0620 08:09:15.931767 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:15.931807 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:15.932381 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:15.932739 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:15.932743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:15.933077 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:15.933358 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:15.933365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.934419 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.934420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:15.935794 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:15.935797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.936305 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.936306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:15.936398 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:15.936959 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:15.936970 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:15.939044 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:15.940996 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:15.941003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:15.941005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:15.941233 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:15.941480 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:15.941481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:15.941481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:15.941575 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:15.941785 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:15.941788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:15.941891 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 08:09:15.941980 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 08:09:15.942109 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260620 08:09:15.942941 14143 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36613
I20260620 08:09:15.943058 14143 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:15.943297 14143 heartbeater.cc:511] Master 127.12.132.254:36613 requested a full tablet report, sending...
I20260620 08:09:15.944048 13938 ts_manager.cc:194] Registered new tserver with Master: ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515)
I20260620 08:09:15.945061 13938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:38367
I20260620 08:09:15.945811 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14011
I20260620 08:09:15.945959 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:15.946156 12819 external_mini_cluster.cc:1428] Setting key 4379e82f83736c0abb2e2880243758e1
I20260620 08:09:15.951583 14149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:15.943976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.193:36939)
0620 08:09:15.944143 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:15.944158 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:15.944251 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:15.944374 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:15.944375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:15.944676 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:15.944763 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:15.944765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.945833 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.945834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:15.946564 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:15.946566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.947095 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.947097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:15.947190 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:15.947730 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:15.947735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:15.948125 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:15.950370 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:15.950371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:15.950371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:15.950540 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:15.950829 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:15.950831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:15.950831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:15.950933 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:15.951295 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:15.951296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:15.951342 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 08:09:15.951420 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 08:09:15.951511 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260620 08:09:15.951685 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:15.944074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36939 (local address 127.12.132.254:36613)
0620 08:09:15.944117 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:15.944119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:15.944231 (+ 112us) server_negotiation.cc:407] Connection header received
0620 08:09:15.944303 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:15.944304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:15.944311 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:15.944366 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:15.944911 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.945687 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.946665 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:15.946974 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:15.947073 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:15.948247 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:15.948248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:15.948249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:15.948256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:15.950238 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.950621 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.950623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.950623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.950686 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:15.951029 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:15.951030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:15.951030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:15.951184 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:15.951265 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:15.951474 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 08:09:15.951574 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:15.975404 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36613
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:16.057055 14152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:16.057250 14152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:16.057272 14152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:16.058688 14152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:16.058743 14152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:16.058764 14152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:16.058782 14152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:16.060281 14152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:36613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:16.060542 14152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:16.060808 14152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:16.065771 14158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:16.065842 14160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:16.065771 14157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:16.066046 14152 server_base.cc:1034] running on GCE node
I20260620 08:09:16.066359 14152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:16.066699 14152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:16.067917 14152 hybrid_clock.cc:648] HybridClock initialized: now 1781942956067874 us; error 60 us; skew 500 ppm
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:16.070719 14152 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:16.071182 14152 webserver.cc:460] Webserver started at http://127.12.132.194:35485/ using document root <none> and password file <none>
I20260620 08:09:16.071374 14152 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:16.071414 14152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:16.071501 14152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:16.072444 14152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "b56790b5ad9e47c49df39f0732159f15"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "5e8928d4dc4a71e722a03947f8d38568"
server_key_iv: "1f915f11ecb00a962a6a321473b7b00a"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.072882 14152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "b56790b5ad9e47c49df39f0732159f15"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "5e8928d4dc4a71e722a03947f8d38568"
server_key_iv: "1f915f11ecb00a962a6a321473b7b00a"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.074084 14152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:16.074844 14167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.075052 14152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:16.075127 14152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "b56790b5ad9e47c49df39f0732159f15"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "5e8928d4dc4a71e722a03947f8d38568"
server_key_iv: "1f915f11ecb00a962a6a321473b7b00a"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.075218 14152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:16.101280 14152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:16.103539 14152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:16.103647 14152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:16.104355 14152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:16.104415 14152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.104452 14152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:16.104519 14152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.113613 14152 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:35705
I20260620 08:09:16.113817 14283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:35705 every 8 connection(s)
I20260620 08:09:16.114055 14152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:16.118413 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.105101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:48121 (local address 127.12.132.254:36613)
0620 08:09:16.105172 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.105175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.106466 (+ 1291us) server_negotiation.cc:407] Connection header received
0620 08:09:16.107274 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.107276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.107290 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.107339 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.108582 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.109427 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.111142 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.111410 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.111469 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.114753 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.114755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.114756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.114763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.116188 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.117134 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.117136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.117136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.117234 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.117535 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.117536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.117537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.117678 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.117770 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.118076 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.118203 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:09:16.118579 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.105142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.194:48121)
0620 08:09:16.106261 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.106321 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.107148 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.107417 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.107423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.107914 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:16.108374 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.108389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.109582 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.109583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.111045 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.111049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.111576 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.111577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.111719 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.112373 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:16.112384 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:16.114648 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:16.116292 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.116299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.116301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.116999 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.117325 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.117326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.117326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.117457 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.117748 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:16.117755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:16.117926 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.118039 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.118233 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":104,"threads_started":1}
I20260620 08:09:16.119196 14284 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36613
I20260620 08:09:16.119282 14284 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:16.119484 14284 heartbeater.cc:511] Master 127.12.132.254:36613 requested a full tablet report, sending...
I20260620 08:09:16.119876 13939 ts_manager.cc:194] Registered new tserver with Master: b56790b5ad9e47c49df39f0732159f15 (127.12.132.194:35705)
I20260620 08:09:16.120406 13939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:48121
I20260620 08:09:16.121395 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14152
I20260620 08:09:16.121527 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:16.121868 12819 external_mini_cluster.cc:1428] Setting key 74a302fef6605bcd088a136dd2f9af42
I20260620 08:09:16.127620 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.120274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:35885 (local address 127.12.132.254:36613)
0620 08:09:16.120317 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.120319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.120418 (+ 99us) server_negotiation.cc:407] Connection header received
0620 08:09:16.120533 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.120534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.120546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.120606 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.121071 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.122041 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.122947 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.123169 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.123243 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.124477 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.124479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.124479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.124488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.126238 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.126689 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.126690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.126690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.126751 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.126973 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.126974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.126974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.127096 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.127153 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.127467 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.127563 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:16.127636 14290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.120191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.194:35885)
0620 08:09:16.120358 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.120368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.120433 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.120612 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.120613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.120849 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:16.120944 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.120946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.122135 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.122136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.122848 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.122850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.123241 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.123242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.123318 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.123900 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:16.123906 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:16.124362 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:16.126338 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.126340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.126340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.126597 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.126819 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.126820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.126820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.126898 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.127167 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:16.127169 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:16.127233 (+ 64us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.127322 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.127467 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:16.151568 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36613
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:16.230654 14293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:16.230876 14293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:16.230901 14293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:16.232038 14293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:16.232079 14293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:16.232098 14293 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:16.232138 14293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:16.233659 14293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:36613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:16.233840 14293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:16.234006 14293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:09:16.238842 14293 server_base.cc:1034] running on GCE node
W20260620 08:09:16.238960 14301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:16.238876 14299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:16.238865 14298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:16.239379 14293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:16.239588 14293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:16.240797 14293 hybrid_clock.cc:648] HybridClock initialized: now 1781942956240782 us; error 47 us; skew 500 ppm
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:16.243223 14293 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:16.243618 14293 webserver.cc:460] Webserver started at http://127.12.132.195:40695/ using document root <none> and password file <none>
I20260620 08:09:16.243882 14293 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:16.243952 14293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:16.244092 14293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:16.245137 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "4fa26b66096f43f58e6231d5665ce7cc"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "512e3f807bcc2d90650e14ed8d2e44db"
server_key_iv: "dc7bd0fcee519c41706561a0be24a50f"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.245483 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "4fa26b66096f43f58e6231d5665ce7cc"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "512e3f807bcc2d90650e14ed8d2e44db"
server_key_iv: "dc7bd0fcee519c41706561a0be24a50f"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.246825 14293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:16.247830 14308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.248070 14293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:16.248170 14293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "4fa26b66096f43f58e6231d5665ce7cc"
format_stamp: "Formatted at 2026-06-20 08:09:16 on dist-test-slave-z0sp"
server_key: "512e3f807bcc2d90650e14ed8d2e44db"
server_key_iv: "dc7bd0fcee519c41706561a0be24a50f"
server_key_version: "encryptionkey@0"
I20260620 08:09:16.248250 14293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:16.277830 14293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:16.280109 14293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:16.280264 14293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:16.280825 14293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:16.280889 14293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.280932 14293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:16.280959 14293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.287885 14293 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:33355
I20260620 08:09:16.288012 14425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:33355 every 8 connection(s)
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:16.288297 14293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:16.291543 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.281360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:44255 (local address 127.12.132.254:36613)
0620 08:09:16.281405 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.281407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.281634 (+ 227us) server_negotiation.cc:407] Connection header received
0620 08:09:16.282148 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.282149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.282161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.282220 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.283072 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.283887 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.285561 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.285762 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.285819 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.289016 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.289017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.289018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.289025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.290440 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.290821 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.290822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.290822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.290869 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.291071 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.291071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.291072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.291164 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.291213 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.291428 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.291481 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:16.291718 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.281366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.195:44255)
0620 08:09:16.281518 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.281550 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.282044 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.282292 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.282295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.282655 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:16.282950 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.282956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.283992 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.283993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.285454 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.285458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.285839 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.285841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.285995 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.286524 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:16.286544 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:16.288908 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:16.290526 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.290531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.290532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.290739 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.290928 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.290929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.290929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.291018 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.291218 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:16.291221 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:16.291320 (+ 99us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.291412 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.291521 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260620 08:09:16.292171 14426 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36613
I20260620 08:09:16.292258 14426 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:16.292447 14426 heartbeater.cc:511] Master 127.12.132.254:36613 requested a full tablet report, sending...
I20260620 08:09:16.292977 13939 ts_manager.cc:194] Registered new tserver with Master: 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355)
I20260620 08:09:16.293351 13939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:44255
I20260620 08:09:16.296934 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14293
I20260620 08:09:16.297035 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:16.297222 12819 external_mini_cluster.cc:1428] Setting key 7b0415aa51e607ba4f243ec7a7046ef1
I20260620 08:09:16.298115 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:16.299757 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.293772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:43351 (local address 127.12.132.254:36613)
0620 08:09:16.293825 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.293828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.294005 (+ 177us) server_negotiation.cc:407] Connection header received
0620 08:09:16.294038 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.294039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.294050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.294123 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.294650 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.295088 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.295895 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.296094 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.296154 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.297064 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.297066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.297066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.297073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.298640 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.298996 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.298997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.298997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.299055 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.299260 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.299262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.299262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.299373 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.299450 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.299606 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.299671 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:09:16.299767 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.293743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.12.132.195:43351)
0620 08:09:16.293908 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.293919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.293997 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.294158 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.294159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.294503 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:16.294558 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.294561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.295239 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.295240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.295800 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.295801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.296164 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.296165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.296218 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.296677 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:16.296681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:16.296933 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:16.298736 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.298736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.298737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.298910 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.299129 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.299130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.299131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.299187 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.299441 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:16.299442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:16.299485 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.299569 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.299668 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:16.324026 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.315887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.12.132.254:36613)
0620 08:09:16.315949 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.315951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.315986 (+ 35us) server_negotiation.cc:407] Connection header received
0620 08:09:16.316054 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.316055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.316062 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.316103 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.316651 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.317103 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.318160 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.318551 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.318808 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.321567 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.321569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.321569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.321577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.322798 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.323261 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.323262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.323262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.323316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.323521 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.323522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.323522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.323626 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.323735 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.323846 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.323928 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 08:09:16.325525 13939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:16.327401 13939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:16.334451 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35705 (local address 127.0.0.1:60868)
0620 08:09:16.330457 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.330495 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.330578 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.330973 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.330976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.330978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.331047 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.331049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.332333 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.332334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.333329 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.333331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334154 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.334155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.334230 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.334244 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.334320 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.334370 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":59,"threads_started":1}
I20260620 08:09:16.334508 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.12.132.194:35705)
0620 08:09:16.330820 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.330824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.330833 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:16.330867 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.330869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.330884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.330969 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.331118 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.332243 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.333422 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334067 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.334144 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.334196 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.334345 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.334386 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":78,"threads_started":1}
I20260620 08:09:16.334769 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35515 (local address 127.0.0.1:33588)
0620 08:09:16.330510 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.330518 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.330578 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.330973 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.330976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.330981 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.331049 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.331051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.332345 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.332346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.333733 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.333735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334568 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.334569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.334645 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.334649 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.334678 (+ 29us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.334710 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":68,"thread_start_us":70,"threads_started":1}
I20260620 08:09:16.334900 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.12.132.195:33355)
0620 08:09:16.330587 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.330592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.330758 (+ 166us) server_negotiation.cc:407] Connection header received
0620 08:09:16.330914 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.330916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.330932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.330983 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.331210 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.331973 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.333713 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334423 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.334494 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.334542 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.334755 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.334795 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260620 08:09:16.334988 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33355 (local address 127.0.0.1:53846)
0620 08:09:16.330457 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.330709 (+ 252us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.330783 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.331069 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.331069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.331071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.331109 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.331110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.332089 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.332090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.333629 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.333632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334582 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.334583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.334649 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.334653 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.334728 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.334775 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":296,"thread_start_us":74,"threads_started":1}
I20260620 08:09:16.335284 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.330623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.12.132.193:35515)
0620 08:09:16.330762 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.330766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.330777 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:16.330834 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.330836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.330852 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.331128 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.331180 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.332238 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.333816 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.334470 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.335042 (+ 572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.335101 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.335129 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.335179 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260620 08:09:16.335733 14358 tablet_service.cc:1467] Processing CreateTablet for tablet 12f2006b23f049db96c5488bca7130bd (DEFAULT_TABLE table=test-table [id=f41dee537ffd468183a4591eb58de24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.335901 14077 tablet_service.cc:1467] Processing CreateTablet for tablet 12f2006b23f049db96c5488bca7130bd (DEFAULT_TABLE table=test-table [id=f41dee537ffd468183a4591eb58de24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.335937 14209 tablet_service.cc:1467] Processing CreateTablet for tablet 12f2006b23f049db96c5488bca7130bd (DEFAULT_TABLE table=test-table [id=f41dee537ffd468183a4591eb58de24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.336117 14358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f2006b23f049db96c5488bca7130bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.336228 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f2006b23f049db96c5488bca7130bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.336228 14077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f2006b23f049db96c5488bca7130bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.338737 14449 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Bootstrap starting.
I20260620 08:09:16.339561 14449 tablet_bootstrap.cc:654] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.339800 14450 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Bootstrap starting.
I20260620 08:09:16.339932 14449 log.cc:824] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:16.340000 14451 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Bootstrap starting.
I20260620 08:09:16.340627 14451 tablet_bootstrap.cc:654] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.340711 14449 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: No bootstrap required, opened a new log
I20260620 08:09:16.340782 14449 ts_tablet_manager.cc:1397] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:09:16.340965 14451 log.cc:824] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:16.341116 14450 tablet_bootstrap.cc:654] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.341428 14450 log.cc:824] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:16.341614 14451 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: No bootstrap required, opened a new log
I20260620 08:09:16.341679 14451 ts_tablet_manager.cc:1397] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:16.342108 14450 tablet_bootstrap.cc:492] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: No bootstrap required, opened a new log
I20260620 08:09:16.342181 14450 ts_tablet_manager.cc:1397] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:16.342871 14449 raft_consensus.cc:348] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.343039 14449 raft_consensus.cc:374] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.343086 14449 raft_consensus.cc:729] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa26b66096f43f58e6231d5665ce7cc, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.343189 14449 consensus_queue.cc:260] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.343431 14449 ts_tablet_manager.cc:1428] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:16.343464 14426 heartbeater.cc:503] Master 127.12.132.254:36613 was elected leader, sending a full tablet report...
I20260620 08:09:16.343701 14451 raft_consensus.cc:348] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.343806 14451 raft_consensus.cc:374] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.343825 14451 raft_consensus.cc:729] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8f1d19331044b8830b10381637ed5b, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.343930 14451 consensus_queue.cc:260] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.344026 14450 raft_consensus.cc:348] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.344168 14451 ts_tablet_manager.cc:1428] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:09:16.344184 14450 raft_consensus.cc:374] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.344215 14143 heartbeater.cc:503] Master 127.12.132.254:36613 was elected leader, sending a full tablet report...
I20260620 08:09:16.344219 14450 raft_consensus.cc:729] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56790b5ad9e47c49df39f0732159f15, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.344340 14450 consensus_queue.cc:260] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.344595 14450 ts_tablet_manager.cc:1428] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260620 08:09:16.344731 14284 heartbeater.cc:503] Master 127.12.132.254:36613 was elected leader, sending a full tablet report...
W20260620 08:09:16.364725 14285 tablet.cc:2369] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:16.413410 14457 raft_consensus.cc:482] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:16.413618 14457 raft_consensus.cc:504] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.414072 14457 leader_election.cc:288] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355), ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515)
I20260620 08:09:16.417423 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.414443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:60071 (local address 127.12.132.195:33355)
0620 08:09:16.414510 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.414512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.414811 (+ 299us) server_negotiation.cc:407] Connection header received
0620 08:09:16.414847 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.414849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.414861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.414957 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.415222 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.415721 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.416728 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.417077 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.417134 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.417140 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.417301 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.417352 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:09:16.417510 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.414420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:58411 (local address 127.12.132.193:35515)
0620 08:09:16.414475 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.414479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.414561 (+ 82us) server_negotiation.cc:407] Connection header received
0620 08:09:16.414628 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.414629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.414642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.414719 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.415060 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.415684 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.416708 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.417090 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.417183 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.417191 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.417383 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.417424 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 08:09:16.417516 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.414370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35515 (local address 127.12.132.194:58411)
0620 08:09:16.414467 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.414480 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.414551 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.414829 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.414832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.414836 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.414939 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.414941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.415771 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.415772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.416623 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.416625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.417195 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.417196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.417301 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.417309 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.417364 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.417399 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 08:09:16.417433 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.414377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33355 (local address 127.12.132.194:60071)
0620 08:09:16.414691 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.414702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.414773 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.415031 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.415032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.415059 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.415133 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.415134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.415874 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.415875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.416624 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.416626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.417155 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.417157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.417233 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.417237 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.417278 (+ 41us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.417309 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":68,"thread_start_us":145,"threads_started":1}
I20260620 08:09:16.417903 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f2006b23f049db96c5488bca7130bd" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b" is_pre_election: true
I20260620 08:09:16.418077 14097 raft_consensus.cc:2454] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 0.
I20260620 08:09:16.418196 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f2006b23f049db96c5488bca7130bd" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa26b66096f43f58e6231d5665ce7cc" is_pre_election: true
I20260620 08:09:16.418336 14379 raft_consensus.cc:2454] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 0.
I20260620 08:09:16.418336 14169 leader_election.cc:302] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b56790b5ad9e47c49df39f0732159f15, ea8f1d19331044b8830b10381637ed5b; no voters:
I20260620 08:09:16.418468 14457 raft_consensus.cc:2781] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:16.418529 14457 raft_consensus.cc:482] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:16.418562 14457 raft_consensus.cc:3037] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.419580 14457 raft_consensus.cc:504] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.419732 14457 leader_election.cc:288] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 election: Requested vote from peers 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355), ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515)
I20260620 08:09:16.419855 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f2006b23f049db96c5488bca7130bd" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa26b66096f43f58e6231d5665ce7cc"
I20260620 08:09:16.419895 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f2006b23f049db96c5488bca7130bd" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b"
I20260620 08:09:16.419938 14379 raft_consensus.cc:3037] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.419963 14097 raft_consensus.cc:3037] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.420717 14097 raft_consensus.cc:2454] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 1.
I20260620 08:09:16.420774 14379 raft_consensus.cc:2454] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 1.
I20260620 08:09:16.420917 14169 leader_election.cc:302] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b56790b5ad9e47c49df39f0732159f15, ea8f1d19331044b8830b10381637ed5b; no voters:
I20260620 08:09:16.421028 14457 raft_consensus.cc:2781] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:16.421221 14457 raft_consensus.cc:686] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 1 LEADER]: Becoming Leader. State: Replica: b56790b5ad9e47c49df39f0732159f15, State: Running, Role: LEADER
I20260620 08:09:16.421319 14457 consensus_queue.cc:237] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } }
I20260620 08:09:16.422165 13939 catalog_manager.cc:5495] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 reported cstate change: term changed from 0 to 1, leader changed from <none> to b56790b5ad9e47c49df39f0732159f15 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "b56790b5ad9e47c49df39f0732159f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } health_report { overall_health: UNKNOWN } } }
W20260620 08:09:16.440594 14144 tablet.cc:2369] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:16.452137 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.449466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.12.132.194:35705)
0620 08:09:16.449537 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.449539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.449549 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:16.449627 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.449628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.449647 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.449729 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:16.449869 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.450346 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.451241 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.451439 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.451518 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.451728 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.451828 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.451913 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.451975 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.451995 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:09:16.453933 14097 raft_consensus.cc:1261] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Refusing update from remote peer b56790b5ad9e47c49df39f0732159f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.453933 14379 raft_consensus.cc:1261] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Refusing update from remote peer b56790b5ad9e47c49df39f0732159f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.454260 14457 consensus_queue.cc:1035] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.454460 14459 consensus_queue.cc:1035] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.460404 13938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:16.460549 13938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:16.462008 13938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Sending DeleteTablet for 3 replicas of tablet 12f2006b23f049db96c5488bca7130bd
I20260620 08:09:16.462508 14077 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f2006b23f049db96c5488bca7130bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:33588
I20260620 08:09:16.463380 13938 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 08:09:16.463459 14209 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f2006b23f049db96c5488bca7130bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60868
I20260620 08:09:16.463507 14471 tablet_replica.cc:330] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: stopping tablet replica
W20260620 08:09:16.463569 13938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:16.463606 14471 raft_consensus.cc:2229] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:16.463722 14471 raft_consensus.cc:2258] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.463850 14358 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f2006b23f049db96c5488bca7130bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:53846
I20260620 08:09:16.464021 14472 tablet_replica.cc:330] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: stopping tablet replica
I20260620 08:09:16.464160 14472 raft_consensus.cc:2229] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:16.464238 14473 tablet_replica.cc:330] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: stopping tablet replica
I20260620 08:09:16.464267 14472 raft_consensus.cc:2258] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.464385 14473 raft_consensus.cc:2229] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:16.464563 14473 raft_consensus.cc:2258] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.464876 14471 ts_tablet_manager.cc:1905] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.465096 14473 ts_tablet_manager.cc:1905] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.465621 14077 tablet_service.cc:1467] Processing CreateTablet for tablet 053e0652881a4986b0bcf7cb033ef78d (DEFAULT_TABLE table=test-table [id=aa2e437387fa47119e91e05d21a5d439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.465621 14358 tablet_service.cc:1467] Processing CreateTablet for tablet 053e0652881a4986b0bcf7cb033ef78d (DEFAULT_TABLE table=test-table [id=aa2e437387fa47119e91e05d21a5d439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.465726 14077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053e0652881a4986b0bcf7cb033ef78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.465726 14358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053e0652881a4986b0bcf7cb033ef78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.466619 14471 ts_tablet_manager.cc:1918] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:16.466682 14471 log.cc:1196] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/12f2006b23f049db96c5488bca7130bd
I20260620 08:09:16.466945 14471 ts_tablet_manager.cc:1939] T 12f2006b23f049db96c5488bca7130bd P ea8f1d19331044b8830b10381637ed5b: Deleting consensus metadata
I20260620 08:09:16.467070 14451 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Bootstrap starting.
I20260620 08:09:16.467365 14449 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Bootstrap starting.
I20260620 08:09:16.467459 14451 tablet_bootstrap.cc:654] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.467774 14449 tablet_bootstrap.cc:654] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.467862 13923 catalog_manager.cc:4841] TS ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515): tablet 12f2006b23f049db96c5488bca7130bd (table test-table [id=f41dee537ffd468183a4591eb58de24c]) successfully deleted
I20260620 08:09:16.468286 14209 tablet_service.cc:1467] Processing CreateTablet for tablet 053e0652881a4986b0bcf7cb033ef78d (DEFAULT_TABLE table=test-table [id=aa2e437387fa47119e91e05d21a5d439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:16.468405 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 053e0652881a4986b0bcf7cb033ef78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.468878 14449 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: No bootstrap required, opened a new log
I20260620 08:09:16.468930 14449 ts_tablet_manager.cc:1397] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:16.469065 14449 raft_consensus.cc:348] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.469141 14449 raft_consensus.cc:374] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.469168 14449 raft_consensus.cc:729] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa26b66096f43f58e6231d5665ce7cc, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.469214 14449 consensus_queue.cc:260] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.469301 14449 ts_tablet_manager.cc:1428] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:16.469477 14473 ts_tablet_manager.cc:1918] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:16.469580 14473 log.cc:1196] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/12f2006b23f049db96c5488bca7130bd
I20260620 08:09:16.469863 14450 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Bootstrap starting.
I20260620 08:09:16.469880 14473 ts_tablet_manager.cc:1939] T 12f2006b23f049db96c5488bca7130bd P 4fa26b66096f43f58e6231d5665ce7cc: Deleting consensus metadata
I20260620 08:09:16.470239 14450 tablet_bootstrap.cc:654] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.470407 13924 catalog_manager.cc:4841] TS 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355): tablet 12f2006b23f049db96c5488bca7130bd (table test-table [id=f41dee537ffd468183a4591eb58de24c]) successfully deleted
I20260620 08:09:16.470889 14450 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: No bootstrap required, opened a new log
I20260620 08:09:16.470938 14450 ts_tablet_manager.cc:1397] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:16.470954 14472 ts_tablet_manager.cc:1905] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.471104 14450 raft_consensus.cc:348] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.471165 14450 raft_consensus.cc:374] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.471186 14450 raft_consensus.cc:729] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56790b5ad9e47c49df39f0732159f15, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.471226 14450 consensus_queue.cc:260] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.471282 14451 tablet_bootstrap.cc:492] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: No bootstrap required, opened a new log
I20260620 08:09:16.471300 14450 ts_tablet_manager.cc:1428] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.471325 14451 ts_tablet_manager.cc:1397] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260620 08:09:16.471655 14451 raft_consensus.cc:348] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.471740 14451 raft_consensus.cc:374] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.471774 14451 raft_consensus.cc:729] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8f1d19331044b8830b10381637ed5b, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.471827 14451 consensus_queue.cc:260] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.471946 14451 ts_tablet_manager.cc:1428] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:16.472572 14472 ts_tablet_manager.cc:1918] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:16.472641 14472 log.cc:1196] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/12f2006b23f049db96c5488bca7130bd
I20260620 08:09:16.472898 14472 ts_tablet_manager.cc:1939] T 12f2006b23f049db96c5488bca7130bd P b56790b5ad9e47c49df39f0732159f15: Deleting consensus metadata
I20260620 08:09:16.473274 13922 catalog_manager.cc:4841] TS b56790b5ad9e47c49df39f0732159f15 (127.12.132.194:35705): tablet 12f2006b23f049db96c5488bca7130bd (table test-table [id=f41dee537ffd468183a4591eb58de24c]) successfully deleted
I20260620 08:09:16.483526 14459 raft_consensus.cc:482] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:16.483603 14459 raft_consensus.cc:504] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.483742 14459 leader_election.cc:288] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515), 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355)
I20260620 08:09:16.483952 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "053e0652881a4986b0bcf7cb033ef78d" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa26b66096f43f58e6231d5665ce7cc" is_pre_election: true
I20260620 08:09:16.483963 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "053e0652881a4986b0bcf7cb033ef78d" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b" is_pre_election: true
I20260620 08:09:16.484038 14379 raft_consensus.cc:2454] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 0.
I20260620 08:09:16.484046 14097 raft_consensus.cc:2454] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 0.
I20260620 08:09:16.484175 14170 leader_election.cc:302] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa26b66096f43f58e6231d5665ce7cc, b56790b5ad9e47c49df39f0732159f15; no voters:
I20260620 08:09:16.484261 14459 raft_consensus.cc:2781] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:16.484298 14459 raft_consensus.cc:482] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:16.484319 14459 raft_consensus.cc:3037] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.485203 14459 raft_consensus.cc:504] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.485335 14459 leader_election.cc:288] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 election: Requested vote from peers ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515), 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355)
I20260620 08:09:16.485503 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "053e0652881a4986b0bcf7cb033ef78d" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa26b66096f43f58e6231d5665ce7cc"
I20260620 08:09:16.485574 14379 raft_consensus.cc:3037] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.485612 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "053e0652881a4986b0bcf7cb033ef78d" candidate_uuid: "b56790b5ad9e47c49df39f0732159f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b"
I20260620 08:09:16.485769 14097 raft_consensus.cc:3037] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.486243 14379 raft_consensus.cc:2454] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 1.
I20260620 08:09:16.486409 14170 leader_election.cc:302] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa26b66096f43f58e6231d5665ce7cc, b56790b5ad9e47c49df39f0732159f15; no voters:
I20260620 08:09:16.486500 14459 raft_consensus.cc:2781] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:16.486542 14459 raft_consensus.cc:686] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 1 LEADER]: Becoming Leader. State: Replica: b56790b5ad9e47c49df39f0732159f15, State: Running, Role: LEADER
I20260620 08:09:16.486601 14459 consensus_queue.cc:237] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.486768 14097 raft_consensus.cc:2454] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b56790b5ad9e47c49df39f0732159f15 in term 1.
I20260620 08:09:16.487135 13938 catalog_manager.cc:5495] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 reported cstate change: term changed from 0 to 1, leader changed from <none> to b56790b5ad9e47c49df39f0732159f15 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "b56790b5ad9e47c49df39f0732159f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } health_report { overall_health: UNKNOWN } } }
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942955, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:16.525283 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.517232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36613 (local address 127.0.0.1:46724)
0620 08:09:16.517408 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.517421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.517500 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.517770 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.517771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.518156 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:16.518221 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.518223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.518988 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.518989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.519834 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.519836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.520207 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.520207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.520260 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.520823 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:16.520828 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:16.522443 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:16.524114 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.524120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.524122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.524364 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.524612 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:16.524613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:16.524613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:16.524681 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:16.524926 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:16.524931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:16.524980 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.525055 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.525164 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260620 08:09:16.525291 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.517317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.12.132.254:36613)
0620 08:09:16.517371 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.517373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.517563 (+ 190us) server_negotiation.cc:407] Connection header received
0620 08:09:16.517618 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.517620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.517639 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.517742 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.518339 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.518910 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.519934 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.520136 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.520189 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.522584 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.522585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.522585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.522591 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.524015 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.524453 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.524453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.524454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.524551 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.524738 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.524739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.524739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.524854 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.524908 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.525080 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.525223 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":26}
I20260620 08:09:16.527539 13939 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:46724:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 08:09:16.527765 13939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:16.529413 14358 tablet_service.cc:1467] Processing CreateTablet for tablet 17d53b54c62e4b209bae9af0975bfef3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2f633894e504cd0b7eb2bc51c0ef3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:16.529436 14077 tablet_service.cc:1467] Processing CreateTablet for tablet 17d53b54c62e4b209bae9af0975bfef3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2f633894e504cd0b7eb2bc51c0ef3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:16.529568 14077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d53b54c62e4b209bae9af0975bfef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.529568 14358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d53b54c62e4b209bae9af0975bfef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.531108 14449 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: Bootstrap starting.
I20260620 08:09:16.531139 14209 tablet_service.cc:1467] Processing CreateTablet for tablet 17d53b54c62e4b209bae9af0975bfef3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2f633894e504cd0b7eb2bc51c0ef3c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:16.531239 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d53b54c62e4b209bae9af0975bfef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:16.531559 14449 tablet_bootstrap.cc:654] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.532166 14450 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: Bootstrap starting.
I20260620 08:09:16.532243 14451 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: Bootstrap starting.
I20260620 08:09:16.532462 14450 tablet_bootstrap.cc:654] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.532650 14451 tablet_bootstrap.cc:654] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:16.533188 14450 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: No bootstrap required, opened a new log
I20260620 08:09:16.533239 14450 ts_tablet_manager.cc:1397] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:16.533345 14449 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: No bootstrap required, opened a new log
I20260620 08:09:16.533387 14449 ts_tablet_manager.cc:1397] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:16.533388 14450 raft_consensus.cc:348] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.533463 14450 raft_consensus.cc:374] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.533488 14450 raft_consensus.cc:729] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56790b5ad9e47c49df39f0732159f15, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.533535 14450 consensus_queue.cc:260] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.533532 14449 raft_consensus.cc:348] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.533609 14449 raft_consensus.cc:374] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.533586 14451 tablet_bootstrap.cc:492] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: No bootstrap required, opened a new log
I20260620 08:09:16.533630 14449 raft_consensus.cc:729] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa26b66096f43f58e6231d5665ce7cc, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.533627 14450 ts_tablet_manager.cc:1428] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:16.533654 14451 ts_tablet_manager.cc:1397] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:16.533672 14449 consensus_queue.cc:260] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.533749 14449 ts_tablet_manager.cc:1428] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.533730 14457 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.533812 14457 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.533845 14451 raft_consensus.cc:348] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.533912 14451 raft_consensus.cc:374] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:16.533936 14451 raft_consensus.cc:729] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8f1d19331044b8830b10381637ed5b, State: Initialized, Role: FOLLOWER
I20260620 08:09:16.533941 14455 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.533977 14451 consensus_queue.cc:260] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.534006 14455 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.534089 14451 ts_tablet_manager.cc:1428] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:16.534116 14456 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.534189 14456 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 08:09:16.538940 14427 tablet.cc:2369] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:16.658742 14455 raft_consensus.cc:482] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:16.658838 14455 raft_consensus.cc:504] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.659236 14455 leader_election.cc:288] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515), b56790b5ad9e47c49df39f0732159f15 (127.12.132.194:35705)
I20260620 08:09:16.663735 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.659637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35705 (local address 127.12.132.195:44417)
0620 08:09:16.659881 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.659892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.659966 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.660167 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.660168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.660169 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.660219 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.660220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.661096 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.661097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.662773 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.662776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.663514 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.663515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.663575 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.663579 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.663631 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.663668 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":52,"thread_start_us":161,"threads_started":1}
I20260620 08:09:16.663800 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.659704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:44417 (local address 127.12.132.194:35705)
0620 08:09:16.659747 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.659749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.660038 (+ 289us) server_negotiation.cc:407] Connection header received
0620 08:09:16.660069 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.660070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.660084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.660160 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.660292 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.660986 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.662907 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.663395 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.663504 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.663515 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.663660 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.663710 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:16.664294 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.659412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35515 (local address 127.12.132.195:59771)
0620 08:09:16.659463 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.659476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.659524 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.659845 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.659846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.659848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:16.659929 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.659931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.660999 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.661001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.663225 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.663227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.664040 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.664042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.664127 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.664130 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.664181 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.664211 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":81}
I20260620 08:09:16.664358 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.659498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:59771 (local address 127.12.132.193:35515)
0620 08:09:16.659628 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.659631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.659667 (+ 36us) server_negotiation.cc:407] Connection header received
0620 08:09:16.659711 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.659713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.659728 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.659827 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:16.660068 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.660896 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.663341 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.663943 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.664036 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.664047 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.664200 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.664272 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 08:09:16.664331 14231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d53b54c62e4b209bae9af0975bfef3" candidate_uuid: "4fa26b66096f43f58e6231d5665ce7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b56790b5ad9e47c49df39f0732159f15" is_pre_election: true
I20260620 08:09:16.664480 14231 raft_consensus.cc:2454] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa26b66096f43f58e6231d5665ce7cc in term 0.
I20260620 08:09:16.664561 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d53b54c62e4b209bae9af0975bfef3" candidate_uuid: "4fa26b66096f43f58e6231d5665ce7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b" is_pre_election: true
I20260620 08:09:16.664636 14097 raft_consensus.cc:2454] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fa26b66096f43f58e6231d5665ce7cc in term 0.
I20260620 08:09:16.664850 14310 leader_election.cc:302] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa26b66096f43f58e6231d5665ce7cc, ea8f1d19331044b8830b10381637ed5b; no voters:
I20260620 08:09:16.664995 14455 raft_consensus.cc:2781] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:16.665036 14455 raft_consensus.cc:482] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:16.665057 14455 raft_consensus.cc:3037] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.665802 14455 raft_consensus.cc:504] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.665946 14455 leader_election.cc:288] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [CANDIDATE]: Term 1 election: Requested vote from peers ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515), b56790b5ad9e47c49df39f0732159f15 (127.12.132.194:35705)
I20260620 08:09:16.666074 14097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d53b54c62e4b209bae9af0975bfef3" candidate_uuid: "4fa26b66096f43f58e6231d5665ce7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea8f1d19331044b8830b10381637ed5b"
I20260620 08:09:16.666136 14097 raft_consensus.cc:3037] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.666136 14231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d53b54c62e4b209bae9af0975bfef3" candidate_uuid: "4fa26b66096f43f58e6231d5665ce7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b56790b5ad9e47c49df39f0732159f15"
I20260620 08:09:16.666204 14231 raft_consensus.cc:3037] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:16.666893 14097 raft_consensus.cc:2454] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa26b66096f43f58e6231d5665ce7cc in term 1.
I20260620 08:09:16.666982 14231 raft_consensus.cc:2454] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fa26b66096f43f58e6231d5665ce7cc in term 1.
I20260620 08:09:16.667126 14309 leader_election.cc:302] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fa26b66096f43f58e6231d5665ce7cc, b56790b5ad9e47c49df39f0732159f15; no voters:
I20260620 08:09:16.667269 14455 raft_consensus.cc:2781] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:16.667446 14455 raft_consensus.cc:686] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [term 1 LEADER]: Becoming Leader. State: Replica: 4fa26b66096f43f58e6231d5665ce7cc, State: Running, Role: LEADER
I20260620 08:09:16.667533 14455 consensus_queue.cc:237] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } }
I20260620 08:09:16.667871 14488 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: TxnStatusTablet state changed. Reason: New leader 4fa26b66096f43f58e6231d5665ce7cc. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.667963 14488 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:16.668201 14490 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 08:09:16.668282 14490 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 08:09:16.668247 13939 catalog_manager.cc:5495] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195). New cstate: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } health_report { overall_health: HEALTHY } } }
I20260620 08:09:16.686267 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.683558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.12.132.195:33355)
0620 08:09:16.683637 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.683639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.683646 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:16.683682 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.683683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.683700 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.683781 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:16.683968 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.684587 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.685402 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.685597 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.685692 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.685893 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.686008 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.686095 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.686148 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.686170 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 08:09:16.686273 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.683239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33355 (local address 127.0.0.1:53858)
0620 08:09:16.683357 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.683407 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.683498 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.683811 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.683812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.683813 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:16.683895 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.683898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.684674 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.684675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.685298 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.685300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.685674 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.685675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.685759 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.685774 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.686077 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.686087 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.686143 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.686199 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260620 08:09:16.687760 14097 raft_consensus.cc:1261] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Refusing update from remote peer 4fa26b66096f43f58e6231d5665ce7cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.688078 14488 consensus_queue.cc:1035] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.688434 14231 raft_consensus.cc:1261] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Refusing update from remote peer 4fa26b66096f43f58e6231d5665ce7cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.688868 14488 consensus_queue.cc:1035] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.688918 14456 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: TxnStatusTablet state changed. Reason: New leader 4fa26b66096f43f58e6231d5665ce7cc. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.689006 14456 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.689726 14457 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: TxnStatusTablet state changed. Reason: New leader 4fa26b66096f43f58e6231d5665ce7cc. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.689870 14457 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.690438 14464 mvcc.cc:205] Tried to move back new op lower bound from 7298838350590431232 to 7298838350510731264. Current Snapshot: MvccSnapshot[applied={T|T < 7298838350590431232}]
I20260620 08:09:16.690719 14455 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.690840 14455 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:16.690765 14488 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.691009 14488 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P 4fa26b66096f43f58e6231d5665ce7cc: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:16.691332 14462 mvcc.cc:205] Tried to move back new op lower bound from 7298838350590431232 to 7298838350510731264. Current Snapshot: MvccSnapshot[applied={T|T < 7298838350590431232}]
I20260620 08:09:16.691610 14456 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.691711 14456 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P ea8f1d19331044b8830b10381637ed5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.691951 14457 tablet_replica.cc:439] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fa26b66096f43f58e6231d5665ce7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 } } peers { permanent_uuid: "b56790b5ad9e47c49df39f0732159f15" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35705 } } peers { permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 } } }
I20260620 08:09:16.692039 14457 tablet_replica.cc:442] T 17d53b54c62e4b209bae9af0975bfef3 P b56790b5ad9e47c49df39f0732159f15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:16.698149 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.695505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:53285 (local address 127.12.132.195:33355)
0620 08:09:16.695557 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.695559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.695686 (+ 127us) server_negotiation.cc:407] Connection header received
0620 08:09:16.695832 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.695833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.695844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.695917 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:16.696045 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.696618 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.697433 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.697641 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.697726 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.697898 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.697930 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.697998 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.698058 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.698075 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:09:16.698171 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.695450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33355 (local address 127.12.132.194:53285)
0620 08:09:16.695613 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.695622 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.695703 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.695922 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.695923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.695924 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:16.695978 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.695979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.696705 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.696706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.697347 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.697348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.697726 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.697727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.697783 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.697797 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.697998 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.698000 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.698053 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.698097 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":56,"threads_started":1}
I20260620 08:09:16.700330 14379 raft_consensus.cc:1261] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Refusing update from remote peer b56790b5ad9e47c49df39f0732159f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.700340 14097 raft_consensus.cc:1261] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Refusing update from remote peer b56790b5ad9e47c49df39f0732159f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:16.700542 14459 consensus_queue.cc:1035] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa26b66096f43f58e6231d5665ce7cc" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.700639 14457 consensus_queue.cc:1035] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea8f1d19331044b8830b10381637ed5b" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:16.702262 14463 mvcc.cc:205] Tried to move back new op lower bound from 7298838350643277824 to 7298838349769371648. Current Snapshot: MvccSnapshot[applied={T|T < 7298838350643277824}]
I20260620 08:09:16.710201 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.707027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35705 (local address 127.12.132.195:35057)
0620 08:09:16.707133 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:16.707145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:16.707201 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:16.707356 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:16.707357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:16.707359 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:16.707433 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.707435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.708447 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.708448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.709268 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:16.709270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.709827 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.709828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:16.709881 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.709894 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.710081 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.710082 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:16.710118 (+ 36us) client_negotiation.cc:240] Negotiation successful
0620 08:09:16.710150 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 08:09:16.710206 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.707073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:35057 (local address 127.12.132.194:35705)
0620 08:09:16.707117 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.707119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.707187 (+ 68us) server_negotiation.cc:407] Connection header received
0620 08:09:16.707265 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.707265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.707274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.707347 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:16.707516 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.708336 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.709420 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.709733 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.709806 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.709985 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:16.710020 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:16.710074 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.710122 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.710144 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:09:16.727954 13939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:16.728067 13939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:16.728717 13939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b15a8a332db348faa8937dbc0112204b: Sending DeleteTablet for 3 replicas of tablet 053e0652881a4986b0bcf7cb033ef78d
I20260620 08:09:16.729001 14077 tablet_service.cc:1514] Processing DeleteTablet for tablet 053e0652881a4986b0bcf7cb033ef78d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:33588
I20260620 08:09:16.729099 14471 tablet_replica.cc:330] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: stopping tablet replica
I20260620 08:09:16.729149 14471 raft_consensus.cc:2229] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:16.729187 14471 raft_consensus.cc:2258] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.729385 14209 tablet_service.cc:1514] Processing DeleteTablet for tablet 053e0652881a4986b0bcf7cb033ef78d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60868
I20260620 08:09:16.729482 14472 tablet_replica.cc:330] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: stopping tablet replica
I20260620 08:09:16.729543 14472 raft_consensus.cc:2229] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:16.729602 14472 raft_consensus.cc:2258] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.729610 14471 ts_tablet_manager.cc:1905] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.729876 14472 ts_tablet_manager.cc:1905] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.731294 14471 ts_tablet_manager.cc:1918] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:16.731362 14471 log.cc:1196] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/053e0652881a4986b0bcf7cb033ef78d
I20260620 08:09:16.731377 14472 ts_tablet_manager.cc:1918] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:16.731426 14472 log.cc:1196] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/053e0652881a4986b0bcf7cb033ef78d
I20260620 08:09:16.731592 14471 ts_tablet_manager.cc:1939] T 053e0652881a4986b0bcf7cb033ef78d P ea8f1d19331044b8830b10381637ed5b: Deleting consensus metadata
I20260620 08:09:16.731632 14472 ts_tablet_manager.cc:1939] T 053e0652881a4986b0bcf7cb033ef78d P b56790b5ad9e47c49df39f0732159f15: Deleting consensus metadata
I20260620 08:09:16.731809 14358 tablet_service.cc:1514] Processing DeleteTablet for tablet 053e0652881a4986b0bcf7cb033ef78d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:16 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:53846
I20260620 08:09:16.731917 14473 tablet_replica.cc:330] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: stopping tablet replica
I20260620 08:09:16.731978 14473 raft_consensus.cc:2229] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:16.732014 14473 raft_consensus.cc:2258] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:16.732312 14473 ts_tablet_manager.cc:1905] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:16.732322 13922 catalog_manager.cc:4841] TS b56790b5ad9e47c49df39f0732159f15 (127.12.132.194:35705): tablet 053e0652881a4986b0bcf7cb033ef78d (table test-table [id=aa2e437387fa47119e91e05d21a5d439]) successfully deleted
I20260620 08:09:16.732384 13923 catalog_manager.cc:4841] TS ea8f1d19331044b8830b10381637ed5b (127.12.132.193:35515): tablet 053e0652881a4986b0bcf7cb033ef78d (table test-table [id=aa2e437387fa47119e91e05d21a5d439]) successfully deleted
I20260620 08:09:16.733858 14473 ts_tablet_manager.cc:1918] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:16.733976 14473 log.cc:1196] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/053e0652881a4986b0bcf7cb033ef78d
I20260620 08:09:16.734231 14473 ts_tablet_manager.cc:1939] T 053e0652881a4986b0bcf7cb033ef78d P 4fa26b66096f43f58e6231d5665ce7cc: Deleting consensus metadata
I20260620 08:09:16.734666 13924 catalog_manager.cc:4841] TS 4fa26b66096f43f58e6231d5665ce7cc (127.12.132.195:33355): tablet 053e0652881a4986b0bcf7cb033ef78d (table test-table [id=aa2e437387fa47119e91e05d21a5d439]) successfully deleted
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[13893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942956, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:16.742055 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.731750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.12.132.193:35515)
0620 08:09:16.731897 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.731899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.732026 (+ 127us) server_negotiation.cc:407] Connection header received
0620 08:09:16.732134 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.732135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.732151 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.732182 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.732987 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.733543 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.734855 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.735050 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.735114 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.738842 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.738859 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.738859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.738869 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.740869 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.741238 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.741242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.741242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.741314 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.741518 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.741519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.741520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.741658 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.741806 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.741876 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.741954 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 08:09:16.742301 14137 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33592
I20260620 08:09:16.750386 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:16.743263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.12.132.254:36613)
0620 08:09:16.743307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:16.743309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:16.743426 (+ 117us) server_negotiation.cc:407] Connection header received
0620 08:09:16.743455 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:16.743456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:16.743466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:16.743533 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:16.744096 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.744899 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.745586 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:16.745885 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:16.745968 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:16.747425 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:16.747426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:16.747427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:16.747434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:16.749098 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.749438 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.749439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.749439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.749507 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:16.749691 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:16.749692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:16.749692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:16.749841 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:16.749941 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:16.750213 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 08:09:16.750292 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 08:09:16.750669 13939 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46738
I20260620 08:09:16.751427 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14011
I20260620 08:09:16.757184 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14152
I20260620 08:09:16.765923 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14293
I20260620 08:09:16.771440 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 13906
2026-06-20T08:09:16Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2361 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[14510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[14510](info): set up 2 sockets
Jun 20 08:09:16 dist-test-slave-z0sp krb5kdc[14510](info): commencing operation
krb5kdc: starting...
W20260620 08:09:17.817291 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:17 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:17.941807 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39921
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:39921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:18.023008 14524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:18.023254 14524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:18.023290 14524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:18.024469 14524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:18.024570 14524 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 08:09:18.024593 14524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:18.024616 14524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:18.024644 14524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:18.024667 14524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:18.026175 14524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:39921
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:18.026454 14524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:18.026687 14524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:18.031952 14532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:18.032014 14530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:18.032020 14529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.032387 14524 server_base.cc:1034] running on GCE node
I20260620 08:09:18.032668 14524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:18.032922 14524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:18.034152 14524 hybrid_clock.cc:648] HybridClock initialized: now 1781942958034113 us; error 49 us; skew 500 ppm
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:18.037509 14524 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:18.037976 14524 webserver.cc:460] Webserver started at http://127.12.132.254:45245/ using document root <none> and password file <none>
I20260620 08:09:18.038236 14524 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:18.038313 14524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:18.038473 14524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:18.039546 14524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "c970b65e86944ea09f7961103e2a8407"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "53d2a5589dde09b53e7e7296997b0f18"
server_key_iv: "c8b9fffc073121e3d8ef9a04b7ab9e26"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.039916 14524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "c970b65e86944ea09f7961103e2a8407"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "53d2a5589dde09b53e7e7296997b0f18"
server_key_iv: "c8b9fffc073121e3d8ef9a04b7ab9e26"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.041426 14524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:18.042554 14539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.042929 14524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.043108 14524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "c970b65e86944ea09f7961103e2a8407"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "53d2a5589dde09b53e7e7296997b0f18"
server_key_iv: "c8b9fffc073121e3d8ef9a04b7ab9e26"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.043226 14524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:18.064828 14524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:18.067149 14524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:18.067313 14524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:18.071949 14524 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:39921
I20260620 08:09:18.072062 14601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:39921 every 8 connection(s)
I20260620 08:09:18.072357 14524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:18.073314 14602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.076098 14602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Bootstrap starting.
I20260620 08:09:18.076862 14602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.077222 14602 log.cc:824] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:18.077841 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14524
I20260620 08:09:18.077945 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:18.078131 12819 external_mini_cluster.cc:1428] Setting key 79f88f72b7f4239f145458bcb3512532
I20260620 08:09:18.078338 14602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: No bootstrap required, opened a new log
I20260620 08:09:18.081550 14602 raft_consensus.cc:348] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } }
I20260620 08:09:18.081777 14602 raft_consensus.cc:374] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.081828 14602 raft_consensus.cc:729] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c970b65e86944ea09f7961103e2a8407, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.081975 14602 consensus_queue.cc:260] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } }
I20260620 08:09:18.082087 14602 raft_consensus.cc:388] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:18.082139 14602 raft_consensus.cc:482] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:18.082208 14602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:18.082969 14602 raft_consensus.cc:504] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } }
I20260620 08:09:18.083114 14602 leader_election.cc:302] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c970b65e86944ea09f7961103e2a8407; no voters:
I20260620 08:09:18.083393 14602 leader_election.cc:288] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:18.083421 14607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:18.083606 14607 raft_consensus.cc:686] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [term 1 LEADER]: Becoming Leader. State: Replica: c970b65e86944ea09f7961103e2a8407, State: Running, Role: LEADER
I20260620 08:09:18.083747 14602 sys_catalog.cc:564] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:18.083709 14607 consensus_queue.cc:237] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } }
I20260620 08:09:18.084163 14608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c970b65e86944ea09f7961103e2a8407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } } }
I20260620 08:09:18.084295 14608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:18.084151 14609 sys_catalog.cc:455] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c970b65e86944ea09f7961103e2a8407. Latest consensus state: current_term: 1 leader_uuid: "c970b65e86944ea09f7961103e2a8407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c970b65e86944ea09f7961103e2a8407" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39921 } } }
I20260620 08:09:18.084702 14609 sys_catalog.cc:458] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:18.084869 14613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:18.085633 14613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.087939 14613 catalog_manager.cc:1347] Generated new cluster ID: d656277f2ce64499abdc22a59badebb2
I20260620 08:09:18.088037 14613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:18.091316 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.078664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.12.132.254:39921)
0620 08:09:18.078928 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.078936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.078963 (+ 27us) server_negotiation.cc:407] Connection header received
0620 08:09:18.081530 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.081538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.081594 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.081783 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.082585 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.083695 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.084733 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.085320 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.085762 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.088139 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.088143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.088143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.088155 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.089803 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.090188 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.090191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.090191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.090295 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.090509 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.090510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.090510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.090732 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.090933 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.090996 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.091134 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":136,"threads_started":1}
I20260620 08:09:18.107836 14613 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:18.108623 14613 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:18.115875 14613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Generated new TSK 0
I20260620 08:09:18.116086 14613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:18.148814 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39921
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:18.246855 14628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:18.247046 14628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:18.247071 14628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:18.248242 14628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:18.248286 14628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:18.248306 14628 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 08:09:18.248324 14628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:18.248342 14628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:18.249845 14628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:18.250059 14628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:18.250299 14628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:18.255167 14637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:18.255159 14633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.255426 14628 server_base.cc:1034] running on GCE node
W20260620 08:09:18.255236 14634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.255725 14628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:18.255977 14628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:18.257143 14628 hybrid_clock.cc:648] HybridClock initialized: now 1781942958257132 us; error 38 us; skew 500 ppm
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:18.259649 14628 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:18.260066 14628 webserver.cc:460] Webserver started at http://127.12.132.193:42191/ using document root <none> and password file <none>
I20260620 08:09:18.260300 14628 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:18.260370 14628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:18.260566 14628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:18.261574 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "981a7e18024c43bdbe866fd3624cfa82"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "cb0ec4447f61c5e25f923205fc525d3f"
server_key_iv: "f3c134cb27171673220837a1fa0eeaf3"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.261932 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "981a7e18024c43bdbe866fd3624cfa82"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "cb0ec4447f61c5e25f923205fc525d3f"
server_key_iv: "f3c134cb27171673220837a1fa0eeaf3"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.263376 14628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:18.264593 14644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.264901 14628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.265036 14628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "981a7e18024c43bdbe866fd3624cfa82"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "cb0ec4447f61c5e25f923205fc525d3f"
server_key_iv: "f3c134cb27171673220837a1fa0eeaf3"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.265192 14628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:18.287438 14628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:18.289885 14628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:18.290202 14628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:18.290920 14628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:18.290982 14628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.291014 14628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:18.291043 14628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.298177 14628 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:39389
I20260620 08:09:18.298257 14760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:39389 every 8 connection(s)
I20260620 08:09:18.298569 14628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:18.301149 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.291371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:41625 (local address 127.12.132.254:39921)
0620 08:09:18.291434 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.291436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.291641 (+ 205us) server_negotiation.cc:407] Connection header received
0620 08:09:18.292172 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.292173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.292189 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.292244 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.293034 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.293542 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.294633 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.294906 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.294998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.298331 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.298333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.298333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.298344 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.299942 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.300309 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.300309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.300310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.300358 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.300582 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.300582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.300582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.300680 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.300734 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.301022 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.301088 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:18.301323 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.291370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.193:41625)
0620 08:09:18.291523 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.291557 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.292067 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.292324 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.292328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.292719 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.292935 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.292940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.293637 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.293638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.294537 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.294539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.295007 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.295009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.295146 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.295630 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.295641 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.298233 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.300028 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.300034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.300036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.300241 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.300403 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.300403 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.300404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.300484 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.300756 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.300763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.300896 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.300983 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.301101 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260620 08:09:18.301818 14761 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39921
I20260620 08:09:18.301898 14761 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:18.302073 14761 heartbeater.cc:511] Master 127.12.132.254:39921 requested a full tablet report, sending...
I20260620 08:09:18.302596 14556 ts_manager.cc:194] Registered new tserver with Master: 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389)
I20260620 08:09:18.303530 14556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:41625
I20260620 08:09:18.305883 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14628
I20260620 08:09:18.305994 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:18.306195 12819 external_mini_cluster.cc:1428] Setting key e124ee6e554befc875b8182fd6787715
I20260620 08:09:18.309599 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.302910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:46693 (local address 127.12.132.254:39921)
0620 08:09:18.302960 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.302961 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.303002 (+ 41us) server_negotiation.cc:407] Connection header received
0620 08:09:18.303063 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.303064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.303071 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.303110 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.303466 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.304195 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.305279 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.305458 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.305513 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.306866 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.306867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.306868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.306874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.308450 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.308935 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.308935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.308935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.308987 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.309189 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.309189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.309189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.309278 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.309325 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.309492 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.309547 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 08:09:18.309741 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.302813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.193:46693)
0620 08:09:18.302939 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.302950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.303011 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.303126 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.303127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.303312 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.303373 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.303375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.304377 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.304379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.305200 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.305205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.305521 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.305522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.305586 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.306103 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.306110 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.306763 (+ 653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.308561 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.308562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.308562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.308851 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.309049 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.309050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.309050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.309123 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.309337 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.309338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.309399 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.309478 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.309603 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:18.333578 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39921
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:18.417052 14770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:18.417250 14770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:18.417274 14770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:18.418344 14770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:18.418375 14770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:18.418396 14770 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 08:09:18.418408 14770 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:18.418424 14770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:18.419766 14770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:18.419955 14770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:18.420130 14770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:18.425750 14776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:18.425803 14777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.425800 14770 server_base.cc:1034] running on GCE node
W20260620 08:09:18.425770 14779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.426177 14770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:18.426460 14770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:18.427631 14770 hybrid_clock.cc:648] HybridClock initialized: now 1781942958427607 us; error 35 us; skew 500 ppm
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:18.430648 14770 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:18.431110 14770 webserver.cc:460] Webserver started at http://127.12.132.194:46697/ using document root <none> and password file <none>
I20260620 08:09:18.431360 14770 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:18.431407 14770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:18.431491 14770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:18.432536 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "fbbee45cabaf466943f0089fd47dc31d"
server_key_iv: "9a67280906e20598bc3962019af4be89"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.432904 14770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "fbbee45cabaf466943f0089fd47dc31d"
server_key_iv: "9a67280906e20598bc3962019af4be89"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.434329 14770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.435384 14786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.435570 14770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.435676 14770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "fbbee45cabaf466943f0089fd47dc31d"
server_key_iv: "9a67280906e20598bc3962019af4be89"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.435750 14770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:18.453253 14770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:18.456197 14770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:18.456372 14770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:18.456914 14770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:18.456979 14770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.457022 14770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:18.457049 14770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.465766 14770 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:42377
I20260620 08:09:18.465866 14901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:42377 every 8 connection(s)
I20260620 08:09:18.466296 14770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:18.468127 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.457509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:46143 (local address 127.12.132.254:39921)
0620 08:09:18.457554 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.457556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.457743 (+ 187us) server_negotiation.cc:407] Connection header received
0620 08:09:18.458353 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.458355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.458370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.458443 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.459315 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.460181 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.461366 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.461859 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.462000 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.465142 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.465143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.465143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.465151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.466853 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.467283 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.467284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.467284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.467333 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.467583 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.467583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.467584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.467686 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.467743 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.467996 (+ 253us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.468063 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:18.468297 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.457436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.194:46143)
0620 08:09:18.457606 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.457655 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.458183 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.458562 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.458567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.458920 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.459212 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.459218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.460296 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.460297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.461246 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.461248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.462080 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.462081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.462203 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.462863 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.462871 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.465026 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.466934 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.466939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.466940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.467203 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.467415 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.467416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.467416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.467520 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.467736 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.467740 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.467873 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.467964 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.468114 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260620 08:09:18.468953 14902 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39921
I20260620 08:09:18.469055 14902 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:18.469275 14902 heartbeater.cc:511] Master 127.12.132.254:39921 requested a full tablet report, sending...
I20260620 08:09:18.469733 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14770
I20260620 08:09:18.469843 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:18.470088 12819 external_mini_cluster.cc:1428] Setting key d194ce7681856c4369da22b5fe57e937
I20260620 08:09:18.470389 14555 ts_manager.cc:194] Registered new tserver with Master: 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377)
I20260620 08:09:18.471086 14555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:46143
I20260620 08:09:18.476676 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.470383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:33905 (local address 127.12.132.254:39921)
0620 08:09:18.470426 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.470427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.470897 (+ 470us) server_negotiation.cc:407] Connection header received
0620 08:09:18.470968 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.470969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.470980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.471035 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.471434 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.471905 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.472595 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.472801 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.472890 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.473982 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.473983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.473984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.473989 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.475481 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.475896 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.475897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.475897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.475947 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.476154 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.476154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.476155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.476277 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.476328 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.476549 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.476619 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:18.476745 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.470429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.194:33905)
0620 08:09:18.470811 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.470822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.470910 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.471041 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.471042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.471268 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.471342 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.471344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.471992 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.471992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.472533 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.472535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.472890 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.472891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.472968 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.473479 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.473484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.473884 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.475563 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.475564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.475565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.475806 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.476017 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.476018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.476019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.476088 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.476343 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.476345 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.476397 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.476484 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.476624 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":318,"threads_started":1}
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:18.500834 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39921
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:18.582280 14911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:18.582473 14911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:18.582499 14911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:18.583632 14911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:18.583700 14911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:18.583761 14911 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 08:09:18.583781 14911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:18.583798 14911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:18.585341 14911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:18.585558 14911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:18.585742 14911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:18.591164 14917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:18.591210 14919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.591414 14911 server_base.cc:1034] running on GCE node
W20260620 08:09:18.591711 14916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:18.592092 14911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:18.592387 14911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:18.593571 14911 hybrid_clock.cc:648] HybridClock initialized: now 1781942958593527 us; error 78 us; skew 500 ppm
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:18.596240 14911 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:18.596789 14911 webserver.cc:460] Webserver started at http://127.12.132.195:34891/ using document root <none> and password file <none>
I20260620 08:09:18.597035 14911 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:18.597103 14911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:18.597254 14911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:18.598265 14911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "6feebf0644834d0e9a3b0ddb06550b63"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "02f64afbd9c47336beab0ea8c7526ce7"
server_key_iv: "6e6aa6397663354d2340513bb3858440"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.598652 14911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "6feebf0644834d0e9a3b0ddb06550b63"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "02f64afbd9c47336beab0ea8c7526ce7"
server_key_iv: "6e6aa6397663354d2340513bb3858440"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.599972 14911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:09:18.600782 14926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.600951 14911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.601042 14911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "6feebf0644834d0e9a3b0ddb06550b63"
format_stamp: "Formatted at 2026-06-20 08:09:18 on dist-test-slave-z0sp"
server_key: "02f64afbd9c47336beab0ea8c7526ce7"
server_key_iv: "6e6aa6397663354d2340513bb3858440"
server_key_version: "encryptionkey@0"
I20260620 08:09:18.601121 14911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:18.618734 14911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:18.620990 14911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:18.621162 14911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:18.621704 14911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:18.621765 14911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.621807 14911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:18.621835 14911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.628958 14911 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:37115
I20260620 08:09:18.629302 14911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:18.629716 15041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:37115 every 8 connection(s)
I20260620 08:09:18.631467 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.622172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.195:58783)
0620 08:09:18.622335 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.622367 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.622861 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.623115 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.623118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.623412 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.623600 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.623604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.624620 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.624621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.625484 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.625486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.625888 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.625889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.626015 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.626536 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.626544 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.628350 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.630190 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.630196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.630197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.630512 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.630695 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.630695 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.630696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.630788 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.631012 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.631016 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.631121 (+ 105us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.631203 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.631315 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260620 08:09:18.631474 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.622181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:58783 (local address 127.12.132.254:39921)
0620 08:09:18.622220 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.622222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.622449 (+ 227us) server_negotiation.cc:407] Connection header received
0620 08:09:18.622973 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.622974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.622987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.623048 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.623699 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.624536 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.625586 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.625793 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.625878 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.628467 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.628469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.628469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.628477 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.630101 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.630585 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.630586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.630586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.630636 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.630858 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.630858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.630859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.630958 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.631007 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.631267 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.631385 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:18.631982 15042 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39921
I20260620 08:09:18.632079 15042 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:18.632242 15042 heartbeater.cc:511] Master 127.12.132.254:39921 requested a full tablet report, sending...
I20260620 08:09:18.632712 14555 ts_manager.cc:194] Registered new tserver with Master: 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:18.633234 14555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:58783
I20260620 08:09:18.636452 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 14911
I20260620 08:09:18.636791 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:18.636997 12819 external_mini_cluster.cc:1428] Setting key 28dc60d1f3ee591c94812482ed7846cd
I20260620 08:09:18.638921 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:18.639524 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.633022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.12.132.195:51849)
0620 08:09:18.633135 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.633145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.633202 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.633319 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.633320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.633514 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.633588 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.633590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.634366 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.634367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.634873 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.634875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.635234 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.635235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.635297 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.635697 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.635702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.635961 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.638210 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.638212 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.638213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.638493 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.638777 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.638778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.638778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.638844 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.639220 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.639222 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.639270 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.639350 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.639450 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":107,"threads_started":1}
I20260620 08:09:18.639586 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.633076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:51849 (local address 127.12.132.254:39921)
0620 08:09:18.633117 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.633119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.633195 (+ 76us) server_negotiation.cc:407] Connection header received
0620 08:09:18.633250 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.633251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.633261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.633310 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.633666 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.634246 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.634975 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.635151 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.635209 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.636052 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.636054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.636054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.636059 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.638085 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.638604 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.638605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.638606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.638671 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.638948 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.638949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.638949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.639095 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.639178 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.639400 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.639504 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.664517 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.656245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.12.132.254:39921)
0620 08:09:18.656326 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.656328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.656336 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:18.656359 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.656360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.656369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.656474 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.657073 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.657553 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.658296 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.658485 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.658561 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.661439 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.661442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.661442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.661452 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.662932 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.663424 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.663425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.663426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.663491 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.663734 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.663735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.663735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.663878 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.664005 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.664256 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.664363 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 08:09:18.666687 14555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:18.668810 14555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:18.675628 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.671867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.12.132.193:39389)
0620 08:09:18.672017 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.672022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.672033 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:18.672077 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.672079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.672095 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.672361 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:18.672412 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673189 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.674580 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.675172 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.675269 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.675325 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.675491 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.675539 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":100,"threads_started":1}
I20260620 08:09:18.675619 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.671599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.0.0.1:56978)
0620 08:09:18.671703 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.671739 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.671829 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.672210 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.672213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.672215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:18.672292 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.672294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673275 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.673276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.674511 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.674513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.675271 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.675272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.675360 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.675373 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.675475 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.675528 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260620 08:09:18.676034 15061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.671630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:42377 (local address 127.0.0.1:49952)
0620 08:09:18.672052 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.672062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.672125 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.672366 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.672367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.672369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:18.672423 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.672425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673763 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.673764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.674785 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.674787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.675823 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.675825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.675885 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.675888 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.675931 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.675968 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260620 08:09:18.676381 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.671878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.12.132.194:42377)
0620 08:09:18.672027 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.672031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.672138 (+ 107us) server_negotiation.cc:407] Connection header received
0620 08:09:18.672184 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.672186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.672203 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.672283 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:18.672568 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673687 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.674889 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.675587 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.675652 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.676218 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.676243 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.676281 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260620 08:09:18.676360 14694 tablet_service.cc:1467] Processing CreateTablet for tablet be4ae2dd8f7d407cb734563c69f336af (DEFAULT_TABLE table=test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:18.676759 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4ae2dd8f7d407cb734563c69f336af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.676990 14836 tablet_service.cc:1467] Processing CreateTablet for tablet be4ae2dd8f7d407cb734563c69f336af (DEFAULT_TABLE table=test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:18.677292 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4ae2dd8f7d407cb734563c69f336af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.679613 15065 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Bootstrap starting.
I20260620 08:09:18.679986 15066 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Bootstrap starting.
I20260620 08:09:18.680027 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.671630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37115 (local address 127.0.0.1:60980)
0620 08:09:18.672200 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.672208 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.672242 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.672426 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.672427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.672429 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:18.672464 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.672466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673771 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.673772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.674756 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.674758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.679793 (+ 5035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.679793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.679877 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.679881 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.679928 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.679980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":38,"thread_start_us":74,"threads_started":1}
I20260620 08:09:18.680285 15065 tablet_bootstrap.cc:654] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.680390 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.672060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.12.132.195:37115)
0620 08:09:18.672235 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.672239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.672248 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:18.672289 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.672291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.672304 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.672372 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:18.672595 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.673661 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.674841 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.675580 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.675652 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.680190 (+ 4538us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.680217 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.680262 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260620 08:09:18.680624 15066 tablet_bootstrap.cc:654] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.680660 15065 log.cc:824] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:18.680943 15066 log.cc:824] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:18.680951 14976 tablet_service.cc:1467] Processing CreateTablet for tablet be4ae2dd8f7d407cb734563c69f336af (DEFAULT_TABLE table=test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:18.681274 14976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4ae2dd8f7d407cb734563c69f336af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.681480 15065 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: No bootstrap required, opened a new log
I20260620 08:09:18.681562 15065 ts_tablet_manager.cc:1397] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:18.681643 15066 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: No bootstrap required, opened a new log
I20260620 08:09:18.681701 15066 ts_tablet_manager.cc:1397] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:18.683555 15065 raft_consensus.cc:348] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.683555 15066 raft_consensus.cc:348] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.683732 15066 raft_consensus.cc:374] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.683732 15065 raft_consensus.cc:374] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.683773 15065 raft_consensus.cc:729] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.683773 15066 raft_consensus.cc:729] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.683883 15065 consensus_queue.cc:260] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.683979 15066 consensus_queue.cc:260] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.684144 15065 ts_tablet_manager.cc:1428] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:18.684149 15069 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Bootstrap starting.
I20260620 08:09:18.684156 14761 heartbeater.cc:503] Master 127.12.132.254:39921 was elected leader, sending a full tablet report...
I20260620 08:09:18.684557 14902 heartbeater.cc:503] Master 127.12.132.254:39921 was elected leader, sending a full tablet report...
I20260620 08:09:18.684540 15066 ts_tablet_manager.cc:1428] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:18.685096 15069 tablet_bootstrap.cc:654] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.685441 15069 log.cc:824] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:18.686100 15069 tablet_bootstrap.cc:492] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: No bootstrap required, opened a new log
I20260620 08:09:18.686188 15069 ts_tablet_manager.cc:1397] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:09:18.687990 15069 raft_consensus.cc:348] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.688133 15069 raft_consensus.cc:374] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.688177 15069 raft_consensus.cc:729] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.688287 15069 consensus_queue.cc:260] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.688592 15069 ts_tablet_manager.cc:1428] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 08:09:18.688623 15042 heartbeater.cc:503] Master 127.12.132.254:39921 was elected leader, sending a full tablet report...
W20260620 08:09:18.717067 14903 tablet.cc:2369] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:09:18.799371 14762 tablet.cc:2369] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:18.856556 15071 raft_consensus.cc:482] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:18.856688 15071 raft_consensus.cc:504] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.857116 15071 leader_election.cc:288] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:18.860360 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.857299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.12.132.194:45085)
0620 08:09:18.857354 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.857366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.857408 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.857590 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.857591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.857593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:18.857752 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.857756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.858692 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.858693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.859439 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.859441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.860105 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.860106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.860193 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.860195 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.860238 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.860266 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:09:18.860387 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.857398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:45085 (local address 127.12.132.193:39389)
0620 08:09:18.857442 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.857444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.857454 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:18.857481 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.857482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.857492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.857555 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:18.857893 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.858583 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.859565 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.860020 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.860080 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.860088 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.860268 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.860317 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:09:18.860437 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.857411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:48547 (local address 127.12.132.195:37115)
0620 08:09:18.857473 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.857475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.857598 (+ 123us) server_negotiation.cc:407] Connection header received
0620 08:09:18.857628 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.857628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.857640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.857787 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:18.858162 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.858673 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.859672 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.860042 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.860133 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.860142 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.860326 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.860363 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:18.860474 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.857332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37115 (local address 127.12.132.194:48547)
0620 08:09:18.857526 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.857535 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.857584 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.857893 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.857895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.857900 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:18.858016 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.858019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.858780 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.858782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.859581 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.859583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.860159 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.860159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.860219 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.860243 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.860311 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.860348 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":116,"threads_started":1}
I20260620 08:09:18.860743 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4ae2dd8f7d407cb734563c69f336af" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82" is_pre_election: true
I20260620 08:09:18.860772 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4ae2dd8f7d407cb734563c69f336af" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63" is_pre_election: true
I20260620 08:09:18.860911 14996 raft_consensus.cc:2454] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 0.
I20260620 08:09:18.860899 14714 raft_consensus.cc:2454] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 0.
I20260620 08:09:18.861107 14790 leader_election.cc:302] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5b4d65bf8a424fb029491349a4f3ec, 6feebf0644834d0e9a3b0ddb06550b63; no voters:
I20260620 08:09:18.861258 15071 raft_consensus.cc:2781] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:18.861334 15071 raft_consensus.cc:482] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:18.861373 15071 raft_consensus.cc:3037] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:18.862053 15071 raft_consensus.cc:504] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.862224 15071 leader_election.cc:288] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 election: Requested vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:18.862394 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4ae2dd8f7d407cb734563c69f336af" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63"
I20260620 08:09:18.862394 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4ae2dd8f7d407cb734563c69f336af" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82"
I20260620 08:09:18.862475 14714 raft_consensus.cc:3037] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:18.862475 14996 raft_consensus.cc:3037] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:18.863284 14996 raft_consensus.cc:2454] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 1.
I20260620 08:09:18.863283 14714 raft_consensus.cc:2454] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 1.
I20260620 08:09:18.863847 14788 leader_election.cc:302] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5b4d65bf8a424fb029491349a4f3ec, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:18.863955 15071 raft_consensus.cc:2781] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:18.864133 15071 raft_consensus.cc:686] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 LEADER]: Becoming Leader. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Running, Role: LEADER
I20260620 08:09:18.864223 15071 consensus_queue.cc:237] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.865018 14556 catalog_manager.cc:5495] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194). New cstate: current_term: 1 leader_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } health_report { overall_health: UNKNOWN } } }
W20260620 08:09:18.879992 15043 tablet.cc:2369] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 08:09:18 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942958, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:18.930161 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.921698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.12.132.254:39921)
0620 08:09:18.921807 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:18.921809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:18.921980 (+ 171us) server_negotiation.cc:407] Connection header received
0620 08:09:18.922022 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:18.922024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:18.922038 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:18.922199 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:18.922732 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.923241 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.924116 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.924310 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.924375 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.927056 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:18.927057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:18.927058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:18.927069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:18.928892 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.929317 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.929318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.929318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.929379 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:18.929582 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:18.929583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:18.929583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:18.929695 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:18.929772 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:18.929944 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 08:09:18.930031 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 08:09:18.930271 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:18.921560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39921 (local address 127.0.0.1:59346)
0620 08:09:18.921854 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:18.921870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:18.921933 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:18.922222 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:18.922223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:18.922553 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:18.922607 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.922610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.923367 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.923369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.923999 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:18.924001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:18.924418 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:18.924419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:18.924474 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:18.925200 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:18.925205 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:18.926940 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:18.929002 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.929010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.929012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.929218 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.929454 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:18.929454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:18.929455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:18.929516 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:18.929762 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:18.929769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:18.929817 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:09:18.929915 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 08:09:18.930114 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":81,"thread_start_us":166,"threads_started":1}
I20260620 08:09:18.932463 14556 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:59346:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 08:09:18.932737 14556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:18.934988 14836 tablet_service.cc:1467] Processing CreateTablet for tablet b197c006c09047508343e7c318458936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=28a9630eb7644cb389be40bd83286ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:18.934986 14976 tablet_service.cc:1467] Processing CreateTablet for tablet b197c006c09047508343e7c318458936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=28a9630eb7644cb389be40bd83286ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:18.934986 14694 tablet_service.cc:1467] Processing CreateTablet for tablet b197c006c09047508343e7c318458936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=28a9630eb7644cb389be40bd83286ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:18.935166 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b197c006c09047508343e7c318458936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.935168 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b197c006c09047508343e7c318458936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.935176 14976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b197c006c09047508343e7c318458936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:18.936532 15066 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: Bootstrap starting.
I20260620 08:09:18.936662 15069 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: Bootstrap starting.
I20260620 08:09:18.937026 15066 tablet_bootstrap.cc:654] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.937216 15069 tablet_bootstrap.cc:654] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.937590 15065 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: Bootstrap starting.
I20260620 08:09:18.937706 15066 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: No bootstrap required, opened a new log
I20260620 08:09:18.937765 15066 ts_tablet_manager.cc:1397] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.937928 15069 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: No bootstrap required, opened a new log
I20260620 08:09:18.937930 15066 raft_consensus.cc:348] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.937973 15069 ts_tablet_manager.cc:1397] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.937996 15066 raft_consensus.cc:374] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.937992 15065 tablet_bootstrap.cc:654] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:18.938011 15066 raft_consensus.cc:729] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.938089 15066 consensus_queue.cc:260] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.938130 15069 raft_consensus.cc:348] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.938201 15069 raft_consensus.cc:374] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.938202 15066 ts_tablet_manager.cc:1428] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.938228 15069 raft_consensus.cc:729] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.938230 15071 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:18.938298 15071 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:18.938282 15069 consensus_queue.cc:260] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.938426 15069 ts_tablet_manager.cc:1428] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:18.938452 15073 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:18.938547 15073 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:18.938563 15065 tablet_bootstrap.cc:492] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: No bootstrap required, opened a new log
I20260620 08:09:18.938598 15065 ts_tablet_manager.cc:1397] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:18.938758 15065 raft_consensus.cc:348] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.938846 15065 raft_consensus.cc:374] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:18.938871 15065 raft_consensus.cc:729] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Initialized, Role: FOLLOWER
I20260620 08:09:18.938922 15065 consensus_queue.cc:260] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:18.939023 15065 ts_tablet_manager.cc:1428] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:18.939348 15070 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:18.939427 15070 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:19.078703 15070 raft_consensus.cc:482] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:19.078852 15070 raft_consensus.cc:504] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.079396 15070 leader_election.cc:288] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:19.082813 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.079706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:44151 (local address 127.12.132.194:42377)
0620 08:09:19.079768 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.079770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.079837 (+ 67us) server_negotiation.cc:407] Connection header received
0620 08:09:19.079890 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.079891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.079900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.079954 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:19.080161 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.080944 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082091 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.082459 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082544 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.082554 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.082712 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.082749 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 08:09:19.083020 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.079535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37115 (local address 127.12.132.193:58915)
0620 08:09:19.079682 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:19.079700 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:19.079797 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:19.079976 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:19.079977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:19.079979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:19.080074 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.080075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.081147 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.081149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.082142 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.082144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.082733 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.082806 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.082815 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 08:09:19.082883 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:09:19.082934 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260620 08:09:19.083179 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.079562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:42377 (local address 127.12.132.193:44151)
0620 08:09:19.079770 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:19.079779 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:19.079831 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:19.079977 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:19.079977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:19.079980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:19.080074 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.080075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.081040 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.081041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.081979 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.081982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.082562 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.082637 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.082640 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:19.082707 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:09:19.082747 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":61,"thread_start_us":57,"threads_started":1}
I20260620 08:09:19.083374 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.079598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:58915 (local address 127.12.132.195:37115)
0620 08:09:19.079645 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.079647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.079784 (+ 137us) server_negotiation.cc:407] Connection header received
0620 08:09:19.079873 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.079874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.079884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.079943 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:19.080161 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.080945 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082272 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.082637 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.082722 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.082731 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.083202 (+ 471us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.083267 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:19.083531 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b197c006c09047508343e7c318458936" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" is_pre_election: true
I20260620 08:09:19.083549 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b197c006c09047508343e7c318458936" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63" is_pre_election: true
I20260620 08:09:19.083634 14996 raft_consensus.cc:2454] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 0.
I20260620 08:09:19.083648 14856 raft_consensus.cc:2454] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 0.
I20260620 08:09:19.083814 14647 leader_election.cc:302] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5b4d65bf8a424fb029491349a4f3ec, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:19.083983 15070 raft_consensus.cc:2781] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:19.084065 15070 raft_consensus.cc:482] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:19.084091 15070 raft_consensus.cc:3037] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.084893 15070 raft_consensus.cc:504] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.085043 15070 leader_election.cc:288] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 election: Requested vote from peers 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:19.085206 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b197c006c09047508343e7c318458936" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63"
I20260620 08:09:19.085299 14996 raft_consensus.cc:3037] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.085263 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b197c006c09047508343e7c318458936" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
I20260620 08:09:19.085369 14856 raft_consensus.cc:3037] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.085922 14996 raft_consensus.cc:2454] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 1.
I20260620 08:09:19.086103 14856 raft_consensus.cc:2454] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 1.
I20260620 08:09:19.086160 14648 leader_election.cc:302] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6feebf0644834d0e9a3b0ddb06550b63, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:19.086243 15070 raft_consensus.cc:2781] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:19.086354 15070 raft_consensus.cc:686] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 LEADER]: Becoming Leader. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Running, Role: LEADER
I20260620 08:09:19.086463 15070 consensus_queue.cc:237] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.086860 15088 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: TxnStatusTablet state changed. Reason: New leader 981a7e18024c43bdbe866fd3624cfa82. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.086961 15088 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:19.087121 15090 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 08:09:19.087236 15090 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 08:09:19.087198 14556 catalog_manager.cc:5495] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:19.124001 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.120228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.0.0.1:56988)
0620 08:09:19.120315 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:19.120327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:19.120370 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:19.120659 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:19.120660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:19.120661 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:19.120717 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.120721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.121709 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.121710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.122699 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.122702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.123173 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.123174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.123287 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.123321 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.123737 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.123746 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 08:09:19.123824 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 08:09:19.123869 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 08:09:19.124434 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.120282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.12.132.193:39389)
0620 08:09:19.120418 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.120421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.120432 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:19.120474 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.120474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.120553 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.120644 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:19.120854 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.121626 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.122796 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.123092 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.123158 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.123462 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.123649 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.123721 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.124304 (+ 583us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.124328 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260620 08:09:19.126075 14856 raft_consensus.cc:1261] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Refusing update from remote peer 981a7e18024c43bdbe866fd3624cfa82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.126075 14996 raft_consensus.cc:1261] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Refusing update from remote peer 981a7e18024c43bdbe866fd3624cfa82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.126364 15088 consensus_queue.cc:1035] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:19.126504 15070 consensus_queue.cc:1035] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:19.127621 15073 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: TxnStatusTablet state changed. Reason: New leader 981a7e18024c43bdbe866fd3624cfa82. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.127740 15073 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:19.127801 15071 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: TxnStatusTablet state changed. Reason: New leader 981a7e18024c43bdbe866fd3624cfa82. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.127892 15071 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:19.128985 15070 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.129074 15070 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:19.129096 15088 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.129166 15088 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 981a7e18024c43bdbe866fd3624cfa82: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:19.129354 15073 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.129455 15073 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 6feebf0644834d0e9a3b0ddb06550b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:19.129525 15071 tablet_replica.cc:439] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } } }
I20260620 08:09:19.129596 15071 tablet_replica.cc:442] T b197c006c09047508343e7c318458936 P 2d5b4d65bf8a424fb029491349a4f3ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:19.129592 15095 mvcc.cc:205] Tried to move back new op lower bound from 7298838360577957888 to 7298838360418619392. Current Snapshot: MvccSnapshot[applied={T|T < 7298838360577957888}]
I20260620 08:09:19.130224 15094 mvcc.cc:205] Tried to move back new op lower bound from 7298838360577957888 to 7298838360418619392. Current Snapshot: MvccSnapshot[applied={T|T < 7298838360577957888}]
I20260620 08:09:19.134636 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.132216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.12.132.194:42377)
0620 08:09:19.132259 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.132261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.132269 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:19.132353 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.132353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.132370 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.132459 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:19.132586 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.133116 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.133877 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.134081 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.134164 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.134332 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.134429 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.134516 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.134538 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.134559 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:19.139755 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.136709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.12.132.194:42265)
0620 08:09:19.136841 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:19.136852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:19.136911 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:19.137028 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:19.137028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:19.137030 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:19.137086 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.137087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.137628 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.137629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.138536 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.138538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.139263 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.139264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.139332 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.139346 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.139526 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.139527 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:19.139570 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 08:09:19.139666 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260620 08:09:19.139853 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.136800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:42265 (local address 127.12.132.193:39389)
0620 08:09:19.136845 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.136847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.136896 (+ 49us) server_negotiation.cc:407] Connection header received
0620 08:09:19.136968 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.136969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.136977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.137015 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:19.137141 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.137559 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.138684 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.139004 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.139247 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.139445 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.139473 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.139513 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.139750 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.139780 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:09:19.142290 14996 raft_consensus.cc:1261] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Refusing update from remote peer 2d5b4d65bf8a424fb029491349a4f3ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.142338 14714 raft_consensus.cc:1261] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Refusing update from remote peer 2d5b4d65bf8a424fb029491349a4f3ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.142606 15071 consensus_queue.cc:1035] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:19.142743 15075 consensus_queue.cc:1035] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:19.144213 15093 mvcc.cc:205] Tried to move back new op lower bound from 7298838360645181440 to 7298838359508275200. Current Snapshot: MvccSnapshot[applied={T|T < 7298838360645181440}]
I20260620 08:09:19.151402 15063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.148192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:46781 (local address 127.12.132.194:42377)
0620 08:09:19.148224 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:19.148225 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:19.148703 (+ 478us) server_negotiation.cc:407] Connection header received
0620 08:09:19.148727 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:19.148728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:19.148737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:19.148793 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:19.148880 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.149589 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.150589 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.150883 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.150959 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.151147 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.151178 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.151235 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:19.151320 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 08:09:19.151338 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:09:19.151432 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:19.148048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:42377 (local address 127.12.132.193:46781)
0620 08:09:19.148606 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:19.148615 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:19.148663 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:19.148794 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:19.148794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:19.148796 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:19.148828 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.148829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.149686 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.149687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.150506 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:19.150509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:19.150985 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:19.150986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:19.151031 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:19.151048 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:19.151262 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:19.151265 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:19.151317 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 08:09:19.151352 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":71,"threads_started":1}
I20260620 08:09:19.160935 14555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:19.161042 14555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:19.161777 14555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Sending DeleteTablet for 3 replicas of tablet be4ae2dd8f7d407cb734563c69f336af
I20260620 08:09:19.162097 14694 tablet_service.cc:1514] Processing DeleteTablet for tablet be4ae2dd8f7d407cb734563c69f336af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:19 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:56978
I20260620 08:09:19.162191 14836 tablet_service.cc:1514] Processing DeleteTablet for tablet be4ae2dd8f7d407cb734563c69f336af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:19 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:49952
I20260620 08:09:19.162503 14555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 08:09:19.162546 14976 tablet_service.cc:1514] Processing DeleteTablet for tablet be4ae2dd8f7d407cb734563c69f336af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:19 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60980
W20260620 08:09:19.162639 14555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:19.162904 15113 tablet_replica.cc:330] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: stopping tablet replica
I20260620 08:09:19.163049 15113 raft_consensus.cc:2229] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:19.163159 15113 raft_consensus.cc:2258] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:19.163625 15113 ts_tablet_manager.cc:1905] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:19.163926 14694 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (DEFAULT_TABLE table=test-table [id=097790f21cf345ffb80ec3855abdab98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:19.164048 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:19.164594 14836 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (DEFAULT_TABLE table=test-table [id=097790f21cf345ffb80ec3855abdab98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:19.164709 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:19.164795 15115 tablet_replica.cc:330] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: stopping tablet replica
I20260620 08:09:19.164929 15115 raft_consensus.cc:2229] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:19.165020 15115 raft_consensus.cc:2258] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:19.165949 15113 ts_tablet_manager.cc:1918] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:19.166085 15066 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Bootstrap starting.
I20260620 08:09:19.166416 15115 ts_tablet_manager.cc:1905] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:19.166510 15066 tablet_bootstrap.cc:654] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:19.167107 14976 tablet_service.cc:1467] Processing CreateTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (DEFAULT_TABLE table=test-table [id=097790f21cf345ffb80ec3855abdab98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:19.167217 14976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:19.167992 15113 log.cc:1196] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/be4ae2dd8f7d407cb734563c69f336af
I20260620 08:09:19.168401 15113 ts_tablet_manager.cc:1939] T be4ae2dd8f7d407cb734563c69f336af P 981a7e18024c43bdbe866fd3624cfa82: Deleting consensus metadata
I20260620 08:09:19.168602 15069 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Bootstrap starting.
I20260620 08:09:19.169008 15069 tablet_bootstrap.cc:654] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:19.169260 15115 ts_tablet_manager.cc:1918] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:19.169495 14541 catalog_manager.cc:4841] TS 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389): tablet be4ae2dd8f7d407cb734563c69f336af (table test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]) successfully deleted
I20260620 08:09:19.169680 15115 log.cc:1196] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/be4ae2dd8f7d407cb734563c69f336af
I20260620 08:09:19.170035 15115 ts_tablet_manager.cc:1939] T be4ae2dd8f7d407cb734563c69f336af P 6feebf0644834d0e9a3b0ddb06550b63: Deleting consensus metadata
I20260620 08:09:19.170449 14543 catalog_manager.cc:4841] TS 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115): tablet be4ae2dd8f7d407cb734563c69f336af (table test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]) successfully deleted
I20260620 08:09:19.170706 15065 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Bootstrap starting.
I20260620 08:09:19.170882 15069 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: No bootstrap required, opened a new log
I20260620 08:09:19.170920 15069 ts_tablet_manager.cc:1397] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:19.171056 15069 raft_consensus.cc:348] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.171116 15069 raft_consensus.cc:374] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:19.171137 15069 raft_consensus.cc:729] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Initialized, Role: FOLLOWER
I20260620 08:09:19.171180 15069 consensus_queue.cc:260] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.171262 15069 ts_tablet_manager.cc:1428] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:19.171334 15065 tablet_bootstrap.cc:654] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:19.171866 15114 tablet_replica.cc:330] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: stopping tablet replica
I20260620 08:09:19.171973 15114 raft_consensus.cc:2229] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:19.172070 15114 raft_consensus.cc:2258] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:19.172533 15066 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: No bootstrap required, opened a new log
I20260620 08:09:19.172580 15066 ts_tablet_manager.cc:1397] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260620 08:09:19.172679 15065 tablet_bootstrap.cc:492] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: No bootstrap required, opened a new log
I20260620 08:09:19.172721 15065 ts_tablet_manager.cc:1397] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:19.172703 15066 raft_consensus.cc:348] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.172765 15066 raft_consensus.cc:374] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:19.172785 15066 raft_consensus.cc:729] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Initialized, Role: FOLLOWER
I20260620 08:09:19.172822 15066 consensus_queue.cc:260] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.172844 15065 raft_consensus.cc:348] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.172895 15066 ts_tablet_manager.cc:1428] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:19.172914 15065 raft_consensus.cc:374] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:19.172936 15065 raft_consensus.cc:729] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Initialized, Role: FOLLOWER
I20260620 08:09:19.172977 15065 consensus_queue.cc:260] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.173058 15065 ts_tablet_manager.cc:1428] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:19.176764 15114 ts_tablet_manager.cc:1905] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:19.178457 15114 ts_tablet_manager.cc:1918] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:19.178543 15114 log.cc:1196] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/be4ae2dd8f7d407cb734563c69f336af
I20260620 08:09:19.178812 15114 ts_tablet_manager.cc:1939] T be4ae2dd8f7d407cb734563c69f336af P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting consensus metadata
I20260620 08:09:19.179316 14542 catalog_manager.cc:4841] TS 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377): tablet be4ae2dd8f7d407cb734563c69f336af (table test-table [id=ebe1fdf9c77148a68f07bdb4ad85e9bf]) successfully deleted
I20260620 08:09:19.361156 15071 raft_consensus.cc:482] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:19.361299 15071 raft_consensus.cc:504] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.361533 15071 leader_election.cc:288] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:19.361894 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9f7f8a18247ad8feb0e98ce1e1db6" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63" is_pre_election: true
I20260620 08:09:19.361850 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9f7f8a18247ad8feb0e98ce1e1db6" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82" is_pre_election: true
I20260620 08:09:19.361992 14996 raft_consensus.cc:2454] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 0.
I20260620 08:09:19.362002 14714 raft_consensus.cc:2454] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 0.
I20260620 08:09:19.362181 14790 leader_election.cc:302] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5b4d65bf8a424fb029491349a4f3ec, 6feebf0644834d0e9a3b0ddb06550b63; no voters:
I20260620 08:09:19.362349 15071 raft_consensus.cc:2781] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:19.362423 15071 raft_consensus.cc:482] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:19.362455 15071 raft_consensus.cc:3037] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.363142 15071 raft_consensus.cc:504] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.363332 15071 leader_election.cc:288] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 election: Requested vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:19.363505 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9f7f8a18247ad8feb0e98ce1e1db6" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82"
I20260620 08:09:19.363503 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a9f7f8a18247ad8feb0e98ce1e1db6" candidate_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63"
I20260620 08:09:19.363596 14714 raft_consensus.cc:3037] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.363575 14996 raft_consensus.cc:3037] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:19.364207 14714 raft_consensus.cc:2454] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 1.
I20260620 08:09:19.364357 14996 raft_consensus.cc:2454] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d5b4d65bf8a424fb029491349a4f3ec in term 1.
I20260620 08:09:19.364524 14790 leader_election.cc:302] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d5b4d65bf8a424fb029491349a4f3ec, 6feebf0644834d0e9a3b0ddb06550b63; no voters:
I20260620 08:09:19.364605 15071 raft_consensus.cc:2781] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:19.364692 15071 raft_consensus.cc:686] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 LEADER]: Becoming Leader. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Running, Role: LEADER
I20260620 08:09:19.364741 15071 consensus_queue.cc:237] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:19.365278 14556 catalog_manager.cc:5495] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194). New cstate: current_term: 1 leader_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:19.403939 14714 raft_consensus.cc:1261] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Refusing update from remote peer 2d5b4d65bf8a424fb029491349a4f3ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.404292 14996 raft_consensus.cc:1261] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Refusing update from remote peer 2d5b4d65bf8a424fb029491349a4f3ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:19.404264 15075 consensus_queue.cc:1035] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:19.404574 15075 consensus_queue.cc:1035] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 08:09:19.409308 15095 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 08:09:20.933696 14555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:20.933801 14555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:20.934664 14555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Sending DeleteTablet for 3 replicas of tablet 02a9f7f8a18247ad8feb0e98ce1e1db6
I20260620 08:09:20.934948 14836 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:20 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:49952
I20260620 08:09:20.934942 14694 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:20 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:56978
I20260620 08:09:20.934979 14976 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:20 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60980
I20260620 08:09:20.935181 15147 tablet_replica.cc:330] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: stopping tablet replica
I20260620 08:09:20.935249 15147 raft_consensus.cc:2229] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:20.935263 15146 tablet_replica.cc:330] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: stopping tablet replica
I20260620 08:09:20.935318 15147 raft_consensus.cc:2258] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:20.935321 15146 raft_consensus.cc:2229] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:20.935389 15146 raft_consensus.cc:2258] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:20.935588 15147 ts_tablet_manager.cc:1905] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:20.935599 15145 tablet_replica.cc:330] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: stopping tablet replica
I20260620 08:09:20.935654 15145 raft_consensus.cc:2229] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:20.935904 15146 ts_tablet_manager.cc:1905] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:20.937306 15147 ts_tablet_manager.cc:1918] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 08:09:20.937395 15147 log.cc:1196] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/02a9f7f8a18247ad8feb0e98ce1e1db6
I20260620 08:09:20.937491 15146 ts_tablet_manager.cc:1918] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 08:09:20.937574 15146 log.cc:1196] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/02a9f7f8a18247ad8feb0e98ce1e1db6
I20260620 08:09:20.937641 15147 ts_tablet_manager.cc:1939] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting consensus metadata
I20260620 08:09:20.937860 15146 ts_tablet_manager.cc:1939] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting consensus metadata
I20260620 08:09:20.938006 14543 catalog_manager.cc:4841] TS 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115): tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (table test-table [id=097790f21cf345ffb80ec3855abdab98]) successfully deleted
I20260620 08:09:20.938218 15145 raft_consensus.cc:2258] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:20.938292 14542 catalog_manager.cc:4841] TS 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377): tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (table test-table [id=097790f21cf345ffb80ec3855abdab98]) successfully deleted
I20260620 08:09:20.938532 15145 ts_tablet_manager.cc:1905] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:20.940120 15145 ts_tablet_manager.cc:1918] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 08:09:20.940187 15145 log.cc:1196] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/02a9f7f8a18247ad8feb0e98ce1e1db6
I20260620 08:09:20.940403 15145 ts_tablet_manager.cc:1939] T 02a9f7f8a18247ad8feb0e98ce1e1db6 P 981a7e18024c43bdbe866fd3624cfa82: Deleting consensus metadata
I20260620 08:09:20.940840 14541 catalog_manager.cc:4841] TS 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389): tablet 02a9f7f8a18247ad8feb0e98ce1e1db6 (table test-table [id=097790f21cf345ffb80ec3855abdab98]) successfully deleted
Jun 20 08:09:20 dist-test-slave-z0sp krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 08:09:20 dist-test-slave-z0sp krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:20.968443 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:20.955641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59356 (local address 127.12.132.254:39921)
0620 08:09:20.955800 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:20.955802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:20.955851 (+ 49us) server_negotiation.cc:407] Connection header received
0620 08:09:20.955877 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:20.955877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:20.955889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:20.955950 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:20.956724 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:20.957234 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:20.958016 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:20.958206 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:20.958269 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:20.965356 (+ 7087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:20.965358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:20.965358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:20.965364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:20.966898 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:20.967346 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:20.967348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:20.967349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:20.967442 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:20.967729 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:20.967730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:20.967733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:20.967973 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:20.968119 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:20.968230 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 08:09:20.968323 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":103,"threads_started":1}
I20260620 08:09:20.970067 14555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:20.970206 14555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:20.971647 14836 tablet_service.cc:1467] Processing CreateTablet for tablet 5b232f581a4a42dcaada44db2663ef75 (DEFAULT_TABLE table=test-table [id=bed8e10e07774547a1b7040280316933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:20.971665 14694 tablet_service.cc:1467] Processing CreateTablet for tablet 5b232f581a4a42dcaada44db2663ef75 (DEFAULT_TABLE table=test-table [id=bed8e10e07774547a1b7040280316933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:20.971665 14976 tablet_service.cc:1467] Processing CreateTablet for tablet 5b232f581a4a42dcaada44db2663ef75 (DEFAULT_TABLE table=test-table [id=bed8e10e07774547a1b7040280316933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:20.971786 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b232f581a4a42dcaada44db2663ef75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:20.971788 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b232f581a4a42dcaada44db2663ef75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:20.971807 14976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b232f581a4a42dcaada44db2663ef75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:20.972980 15156 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Bootstrap starting.
I20260620 08:09:20.973202 15158 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Bootstrap starting.
I20260620 08:09:20.973315 15157 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Bootstrap starting.
I20260620 08:09:20.973443 15156 tablet_bootstrap.cc:654] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:20.973625 15158 tablet_bootstrap.cc:654] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:20.973704 15157 tablet_bootstrap.cc:654] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:20.974146 15158 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: No bootstrap required, opened a new log
I20260620 08:09:20.974189 15158 ts_tablet_manager.cc:1397] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:20.974097 15156 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: No bootstrap required, opened a new log
I20260620 08:09:20.974249 15156 ts_tablet_manager.cc:1397] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:20.974282 15158 raft_consensus.cc:348] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.974366 15158 raft_consensus.cc:374] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:20.974390 15158 raft_consensus.cc:729] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Initialized, Role: FOLLOWER
I20260620 08:09:20.974395 15156 raft_consensus.cc:348] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.974471 15156 raft_consensus.cc:374] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:20.974444 15158 consensus_queue.cc:260] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.974499 15156 raft_consensus.cc:729] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Initialized, Role: FOLLOWER
I20260620 08:09:20.974542 15156 consensus_queue.cc:260] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.974571 15158 ts_tablet_manager.cc:1428] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:20.974730 15156 ts_tablet_manager.cc:1428] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:20.975430 15157 tablet_bootstrap.cc:492] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: No bootstrap required, opened a new log
I20260620 08:09:20.975487 15157 ts_tablet_manager.cc:1397] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:20.975603 15157 raft_consensus.cc:348] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.975666 15157 raft_consensus.cc:374] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:20.975710 15157 raft_consensus.cc:729] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Initialized, Role: FOLLOWER
I20260620 08:09:20.975807 15157 consensus_queue.cc:260] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:20.975926 15157 ts_tablet_manager.cc:1428] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:21.098218 15162 raft_consensus.cc:482] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:21.098356 15162 raft_consensus.cc:504] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.098878 15162 leader_election.cc:288] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377)
I20260620 08:09:21.103389 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.099219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:59611 (local address 127.12.132.193:39389)
0620 08:09:21.099423 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.099426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.099465 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:09:21.099618 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.099619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.099636 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.099759 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:21.100089 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.100739 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.101933 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.102642 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.102720 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.102731 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.103189 (+ 458us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.103238 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260620 08:09:21.103571 15163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.099142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.12.132.195:59611)
0620 08:09:21.099384 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:21.099398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:21.099482 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:21.099797 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:21.099798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:21.099801 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:21.099928 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.099932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.100862 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.100864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.101831 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.101834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.102809 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.102811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.102903 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.102908 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:21.102960 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 08:09:21.103029 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":154,"threads_started":1}
I20260620 08:09:21.103914 15164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.099191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:42377 (local address 127.12.132.195:40163)
0620 08:09:21.099386 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:21.099398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:21.099482 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:21.099797 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:21.099798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:21.099803 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:21.100692 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.100696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.101455 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.101457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.102538 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.102541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.103547 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.103548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.103679 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.103684 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:21.103742 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:09:21.103789 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":30,"thread_start_us":121,"threads_started":1}
I20260620 08:09:21.104192 15166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.099259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:40163 (local address 127.12.132.194:42377)
0620 08:09:21.099423 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.099426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.099465 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:09:21.099627 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.099628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.099641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.099753 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:21.100783 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.101365 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.102661 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.103282 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.103394 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.103437 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.104027 (+ 590us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.104104 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260620 08:09:21.104354 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b232f581a4a42dcaada44db2663ef75" candidate_uuid: "6feebf0644834d0e9a3b0ddb06550b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82" is_pre_election: true
I20260620 08:09:21.104516 14714 raft_consensus.cc:2454] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6feebf0644834d0e9a3b0ddb06550b63 in term 0.
I20260620 08:09:21.105731 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b232f581a4a42dcaada44db2663ef75" candidate_uuid: "6feebf0644834d0e9a3b0ddb06550b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" is_pre_election: true
I20260620 08:09:21.105728 14928 leader_election.cc:302] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6feebf0644834d0e9a3b0ddb06550b63, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:21.105837 14856 raft_consensus.cc:2454] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6feebf0644834d0e9a3b0ddb06550b63 in term 0.
I20260620 08:09:21.105890 15162 raft_consensus.cc:2781] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:21.105926 15162 raft_consensus.cc:482] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:21.105948 15162 raft_consensus.cc:3037] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.107029 15162 raft_consensus.cc:504] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.107191 15162 leader_election.cc:288] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [CANDIDATE]: Term 1 election: Requested vote from peers 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389), 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377)
I20260620 08:09:21.107339 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b232f581a4a42dcaada44db2663ef75" candidate_uuid: "6feebf0644834d0e9a3b0ddb06550b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
I20260620 08:09:21.107412 14856 raft_consensus.cc:3037] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.107425 14714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b232f581a4a42dcaada44db2663ef75" candidate_uuid: "6feebf0644834d0e9a3b0ddb06550b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981a7e18024c43bdbe866fd3624cfa82"
I20260620 08:09:21.107513 14714 raft_consensus.cc:3037] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.108865 14856 raft_consensus.cc:2454] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6feebf0644834d0e9a3b0ddb06550b63 in term 1.
I20260620 08:09:21.108865 14714 raft_consensus.cc:2454] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6feebf0644834d0e9a3b0ddb06550b63 in term 1.
I20260620 08:09:21.109267 14928 leader_election.cc:302] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6feebf0644834d0e9a3b0ddb06550b63, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:21.109411 15162 raft_consensus.cc:2781] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:21.109609 15162 raft_consensus.cc:686] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 LEADER]: Becoming Leader. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Running, Role: LEADER
I20260620 08:09:21.109702 15162 consensus_queue.cc:237] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.110658 14556 catalog_manager.cc:5495] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "6feebf0644834d0e9a3b0ddb06550b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } health_report { overall_health: HEALTHY } } }
I20260620 08:09:21.127287 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.123982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.12.132.195:37115)
0620 08:09:21.124117 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.124119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.124128 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:21.124173 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.124174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.124200 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.124290 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:21.124430 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.125271 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.126218 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.126488 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.126568 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.126773 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.126977 (+ 204us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.127084 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.127151 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.127176 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260620 08:09:21.132952 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.130025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:36339 (local address 127.12.132.193:39389)
0620 08:09:21.130070 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.130072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.130081 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:21.130106 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.130106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.130120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.130180 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:21.130347 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.131031 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.132223 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.132451 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.132565 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.132742 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.132773 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.132823 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.132882 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.132900 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:21.132970 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.129835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.12.132.195:36339)
0620 08:09:21.129958 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:21.129991 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:21.130048 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:21.130201 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:21.130202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:21.130204 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:21.130262 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.130263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.131134 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.131136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.132115 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.132117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.132566 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.132567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.132636 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.132649 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.132841 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.132842 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:21.132878 (+ 36us) client_negotiation.cc:240] Negotiation successful
0620 08:09:21.132907 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260620 08:09:21.134896 14856 raft_consensus.cc:1261] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Refusing update from remote peer 6feebf0644834d0e9a3b0ddb06550b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:21.134918 14714 raft_consensus.cc:1261] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Refusing update from remote peer 6feebf0644834d0e9a3b0ddb06550b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:21.135134 15167 consensus_queue.cc:1035] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:21.135264 15162 consensus_queue.cc:1035] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:21.141942 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.139594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:58973 (local address 127.12.132.195:37115)
0620 08:09:21.139646 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.139648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.139785 (+ 137us) server_negotiation.cc:407] Connection header received
0620 08:09:21.139809 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.139810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.139821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.139919 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:21.140022 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.140549 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.141287 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.141492 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.141578 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.141739 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.141768 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.141814 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.141861 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.141880 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 08:09:21.141990 15177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.139548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37115 (local address 127.12.132.193:58973)
0620 08:09:21.139660 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:21.139670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:21.139723 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:21.139924 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:21.139925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:21.139927 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:21.139970 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.139971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.140617 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.140618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.141183 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.141185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.141579 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.141580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.141645 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.141649 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.141814 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.141815 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:21.141860 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 08:09:21.141902 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":72,"threads_started":1}
I20260620 08:09:21.149576 14555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:21.149677 14555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:21.150324 14555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Sending DeleteTablet for 3 replicas of tablet 5b232f581a4a42dcaada44db2663ef75
I20260620 08:09:21.150544 14836 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b232f581a4a42dcaada44db2663ef75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:21 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:49952
I20260620 08:09:21.150549 14976 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b232f581a4a42dcaada44db2663ef75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:21 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60980
I20260620 08:09:21.150636 15147 tablet_replica.cc:330] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: stopping tablet replica
I20260620 08:09:21.150641 15146 tablet_replica.cc:330] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: stopping tablet replica
I20260620 08:09:21.150691 15147 raft_consensus.cc:2229] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:21.150699 15146 raft_consensus.cc:2229] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:21.150734 15146 raft_consensus.cc:2258] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:21.150801 15147 raft_consensus.cc:2258] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:21.150820 14694 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b232f581a4a42dcaada44db2663ef75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:21 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:56978
I20260620 08:09:21.150894 15145 tablet_replica.cc:330] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: stopping tablet replica
I20260620 08:09:21.150956 15145 raft_consensus.cc:2229] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:21.151005 15145 raft_consensus.cc:2258] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:21.151165 15147 ts_tablet_manager.cc:1905] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:21.151295 15145 ts_tablet_manager.cc:1905] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:21.152973 15146 ts_tablet_manager.cc:1905] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:21.153174 15145 ts_tablet_manager.cc:1918] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:21.153218 15145 log.cc:1196] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/5b232f581a4a42dcaada44db2663ef75
I20260620 08:09:21.153425 15145 ts_tablet_manager.cc:1939] T 5b232f581a4a42dcaada44db2663ef75 P 981a7e18024c43bdbe866fd3624cfa82: Deleting consensus metadata
I20260620 08:09:21.153275 14555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:21.153510 14555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:21.153709 14541 catalog_manager.cc:4841] TS 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389): tablet 5b232f581a4a42dcaada44db2663ef75 (table test-table [id=bed8e10e07774547a1b7040280316933]) successfully deleted
I20260620 08:09:21.154425 15146 ts_tablet_manager.cc:1918] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:21.154482 15146 log.cc:1196] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/5b232f581a4a42dcaada44db2663ef75
I20260620 08:09:21.154647 15146 ts_tablet_manager.cc:1939] T 5b232f581a4a42dcaada44db2663ef75 P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting consensus metadata
I20260620 08:09:21.155028 14542 catalog_manager.cc:4841] TS 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377): tablet 5b232f581a4a42dcaada44db2663ef75 (table test-table [id=bed8e10e07774547a1b7040280316933]) successfully deleted
I20260620 08:09:21.155690 15147 ts_tablet_manager.cc:1918] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:21.155754 15147 log.cc:1196] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/5b232f581a4a42dcaada44db2663ef75
I20260620 08:09:21.155961 15147 ts_tablet_manager.cc:1939] T 5b232f581a4a42dcaada44db2663ef75 P 6feebf0644834d0e9a3b0ddb06550b63: Deleting consensus metadata
I20260620 08:09:21.156281 14543 catalog_manager.cc:4841] TS 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115): tablet 5b232f581a4a42dcaada44db2663ef75 (table test-table [id=bed8e10e07774547a1b7040280316933]) successfully deleted
I20260620 08:09:21.156482 14836 tablet_service.cc:1467] Processing CreateTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa (DEFAULT_TABLE table=test-table [id=1b9a03dd3fc74de4bca58a5b75222207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:21.156523 14976 tablet_service.cc:1467] Processing CreateTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa (DEFAULT_TABLE table=test-table [id=1b9a03dd3fc74de4bca58a5b75222207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:21.156616 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba334d5c9574a0c81e6bf99142aa1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:21.156617 14976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba334d5c9574a0c81e6bf99142aa1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:21.156615 14694 tablet_service.cc:1467] Processing CreateTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa (DEFAULT_TABLE table=test-table [id=1b9a03dd3fc74de4bca58a5b75222207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:21.156719 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba334d5c9574a0c81e6bf99142aa1aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:21.157717 15157 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Bootstrap starting.
I20260620 08:09:21.157795 15156 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Bootstrap starting.
I20260620 08:09:21.157897 15158 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: Bootstrap starting.
I20260620 08:09:21.158051 15157 tablet_bootstrap.cc:654] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:21.158171 15156 tablet_bootstrap.cc:654] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:21.158234 15158 tablet_bootstrap.cc:654] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:21.158805 15156 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: No bootstrap required, opened a new log
I20260620 08:09:21.158864 15156 ts_tablet_manager.cc:1397] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:21.158972 15158 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: No bootstrap required, opened a new log
I20260620 08:09:21.159024 15158 ts_tablet_manager.cc:1397] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:21.159040 15156 raft_consensus.cc:348] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.159112 15156 raft_consensus.cc:374] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:21.159143 15156 raft_consensus.cc:729] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6feebf0644834d0e9a3b0ddb06550b63, State: Initialized, Role: FOLLOWER
I20260620 08:09:21.159119 15158 raft_consensus.cc:348] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.159180 15158 raft_consensus.cc:374] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:21.159201 15158 raft_consensus.cc:729] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Initialized, Role: FOLLOWER
I20260620 08:09:21.159181 15156 consensus_queue.cc:260] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.159236 15158 consensus_queue.cc:260] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.159292 15156 ts_tablet_manager.cc:1428] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:21.159324 15158 ts_tablet_manager.cc:1428] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:21.159833 15157 tablet_bootstrap.cc:492] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: No bootstrap required, opened a new log
I20260620 08:09:21.159896 15157 ts_tablet_manager.cc:1397] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:21.159998 15157 raft_consensus.cc:348] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.160064 15157 raft_consensus.cc:374] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:21.160079 15157 raft_consensus.cc:729] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5b4d65bf8a424fb029491349a4f3ec, State: Initialized, Role: FOLLOWER
I20260620 08:09:21.160130 15157 consensus_queue.cc:260] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.160225 15157 ts_tablet_manager.cc:1428] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:21.302348 15130 raft_consensus.cc:482] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:21.302438 15130 raft_consensus.cc:504] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.302604 15130 leader_election.cc:288] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:21.302801 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba334d5c9574a0c81e6bf99142aa1aa" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63" is_pre_election: true
I20260620 08:09:21.302788 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba334d5c9574a0c81e6bf99142aa1aa" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" is_pre_election: true
I20260620 08:09:21.302894 14856 raft_consensus.cc:2454] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 0.
I20260620 08:09:21.302894 14996 raft_consensus.cc:2454] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 0.
I20260620 08:09:21.303045 14648 leader_election.cc:302] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6feebf0644834d0e9a3b0ddb06550b63, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:21.303151 15130 raft_consensus.cc:2781] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:21.303196 15130 raft_consensus.cc:482] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:21.303220 15130 raft_consensus.cc:3037] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.303974 15130 raft_consensus.cc:504] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.304134 15130 leader_election.cc:288] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 election: Requested vote from peers 2d5b4d65bf8a424fb029491349a4f3ec (127.12.132.194:42377), 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115)
I20260620 08:09:21.304308 14996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba334d5c9574a0c81e6bf99142aa1aa" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6feebf0644834d0e9a3b0ddb06550b63"
I20260620 08:09:21.304380 14996 raft_consensus.cc:3037] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.304360 14856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba334d5c9574a0c81e6bf99142aa1aa" candidate_uuid: "981a7e18024c43bdbe866fd3624cfa82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d5b4d65bf8a424fb029491349a4f3ec"
I20260620 08:09:21.304636 14856 raft_consensus.cc:3037] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:21.305073 14996 raft_consensus.cc:2454] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 1.
I20260620 08:09:21.305261 14648 leader_election.cc:302] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6feebf0644834d0e9a3b0ddb06550b63, 981a7e18024c43bdbe866fd3624cfa82; no voters:
I20260620 08:09:21.305379 15130 raft_consensus.cc:2781] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:21.305457 15130 raft_consensus.cc:686] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 1 LEADER]: Becoming Leader. State: Replica: 981a7e18024c43bdbe866fd3624cfa82, State: Running, Role: LEADER
I20260620 08:09:21.305514 15130 consensus_queue.cc:237] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } }
I20260620 08:09:21.305738 14856 raft_consensus.cc:2454] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 981a7e18024c43bdbe866fd3624cfa82 in term 1.
I20260620 08:09:21.306185 14556 catalog_manager.cc:5495] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "981a7e18024c43bdbe866fd3624cfa82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "981a7e18024c43bdbe866fd3624cfa82" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 39389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:21.346753 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.344407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.12.132.193:39389)
0620 08:09:21.344523 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.344526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.344534 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:21.344557 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.344558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.344571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.344632 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:21.344737 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.345213 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.346047 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.346314 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.346378 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.346520 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.346551 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.346598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.346646 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.346668 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 08:09:21.351466 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.349377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:34345 (local address 127.12.132.193:39389)
0620 08:09:21.349407 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:21.349409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:21.349416 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:21.349440 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:21.349440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:21.349446 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:21.349492 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:21.349576 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.349991 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.350861 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.351044 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.351104 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.351215 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.351236 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.351271 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:21.351373 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 08:09:21.351385 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 08:09:21.351534 15177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:21.349250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.12.132.193:34345)
0620 08:09:21.349313 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:21.349324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:21.349370 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:21.349504 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:21.349505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:21.349506 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:21.349538 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.349539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.350108 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.350110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.350762 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:21.350765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:21.351115 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:21.351116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:21.351156 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:21.351161 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:21.351306 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:21.351308 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:09:21.351371 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 08:09:21.351409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 08:09:21.353904 14856 raft_consensus.cc:1261] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Refusing update from remote peer 981a7e18024c43bdbe866fd3624cfa82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:21.353930 14996 raft_consensus.cc:1261] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Refusing update from remote peer 981a7e18024c43bdbe866fd3624cfa82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:21.354135 15125 consensus_queue.cc:1035] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d5b4d65bf8a424fb029491349a4f3ec" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:21.354226 15130 consensus_queue.cc:1035] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6feebf0644834d0e9a3b0ddb06550b63" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 08:09:21.358325 15140 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 08:09:22.881510 14555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:22.881620 14555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:22.882440 14555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c970b65e86944ea09f7961103e2a8407: Sending DeleteTablet for 3 replicas of tablet bba334d5c9574a0c81e6bf99142aa1aa
I20260620 08:09:22.882733 14976 tablet_service.cc:1514] Processing DeleteTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:22 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60980
I20260620 08:09:22.882786 14836 tablet_service.cc:1514] Processing DeleteTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:22 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:49952
I20260620 08:09:22.882771 14693 tablet_service.cc:1514] Processing DeleteTablet for tablet bba334d5c9574a0c81e6bf99142aa1aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:22 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:56978
I20260620 08:09:22.882983 15209 tablet_replica.cc:330] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82: stopping tablet replica
I20260620 08:09:22.883046 15209 raft_consensus.cc:2229] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:22.883109 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14628
I20260620 08:09:22.883164 15209 raft_consensus.cc:2258] T bba334d5c9574a0c81e6bf99142aa1aa P 981a7e18024c43bdbe866fd3624cfa82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:22.888620 15208 tablet_replica.cc:330] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: stopping tablet replica
I20260620 08:09:22.888701 15208 raft_consensus.cc:2229] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:22.888743 15208 raft_consensus.cc:2258] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:22.888855 15207 tablet_replica.cc:330] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: stopping tablet replica
I20260620 08:09:22.888963 15207 raft_consensus.cc:2229] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:22.889015 15207 raft_consensus.cc:2258] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:22.889035 15208 ts_tablet_manager.cc:1905] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:22.889249 15207 ts_tablet_manager.cc:1905] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:22.890723 15207 ts_tablet_manager.cc:1918] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 08:09:22.890820 15207 log.cc:1196] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/bba334d5c9574a0c81e6bf99142aa1aa
W20260620 08:09:22.890925 14541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:39389 (error 108)
I20260620 08:09:22.891049 15208 ts_tablet_manager.cc:1918] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 08:09:22.891088 15207 ts_tablet_manager.cc:1939] T bba334d5c9574a0c81e6bf99142aa1aa P 6feebf0644834d0e9a3b0ddb06550b63: Deleting consensus metadata
I20260620 08:09:22.891110 15208 log.cc:1196] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/bba334d5c9574a0c81e6bf99142aa1aa
I20260620 08:09:22.891350 15208 ts_tablet_manager.cc:1939] T bba334d5c9574a0c81e6bf99142aa1aa P 2d5b4d65bf8a424fb029491349a4f3ec: Deleting consensus metadata
I20260620 08:09:22.891543 14543 catalog_manager.cc:4841] TS 6feebf0644834d0e9a3b0ddb06550b63 (127.12.132.195:37115): tablet bba334d5c9574a0c81e6bf99142aa1aa (table test-table [id=1b9a03dd3fc74de4bca58a5b75222207]) successfully deleted
I20260620 08:09:22.891629 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:22.891415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:39389 (local address 127.0.0.1:57000)
0620 08:09:22.891538 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:22.891587 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:39389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260620 08:09:22.891626 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14770
W20260620 08:09:22.893008 14541 catalog_manager.cc:4584] TS 981a7e18024c43bdbe866fd3624cfa82 (127.12.132.193:39389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bba334d5c9574a0c81e6bf99142aa1aa: Network error: Client connection negotiation failed: client connection to 127.12.132.193:39389: connect: Connection refused (error 111)
I20260620 08:09:22.897348 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:22.897239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:42377 (local address 127.0.0.1:49974)
0620 08:09:22.897291 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:22.897316 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:42377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:09:22.897416 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14911
I20260620 08:09:22.905632 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 14524
2026-06-20T08:09:22Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6136 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:22 dist-test-slave-z0sp krb5kdc[15214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:22 dist-test-slave-z0sp krb5kdc[15214](info): set up 2 sockets
Jun 20 08:09:22 dist-test-slave-z0sp krb5kdc[15214](info): commencing operation
krb5kdc: starting...
W20260620 08:09:23.956880 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:24.085103 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:46857
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:24.172297 15227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:24.172557 15227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:24.172623 15227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:24.173791 15227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:24.173862 15227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:24.173888 15227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:24.173916 15227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:24.173954 15227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:24.175429 15227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:24.175710 15227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:24.175957 15227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:24.181038 15232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.181059 15233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.181113 15227 server_base.cc:1034] running on GCE node
W20260620 08:09:24.181041 15235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.181593 15227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:24.181890 15227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:24.183087 15227 hybrid_clock.cc:648] HybridClock initialized: now 1781942964183071 us; error 43 us; skew 500 ppm
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:24.186280 15227 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:24.186764 15227 webserver.cc:460] Webserver started at http://127.12.132.254:41289/ using document root <none> and password file <none>
I20260620 08:09:24.187048 15227 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:24.187206 15227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:24.187551 15227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:24.189018 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "a3c286beca79475cad9273b326a75741"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3757635c940de7658760f7e3f00adb13"
server_key_iv: "156a6d723dbaf66daff18c229c2f632b"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.189433 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "a3c286beca79475cad9273b326a75741"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3757635c940de7658760f7e3f00adb13"
server_key_iv: "156a6d723dbaf66daff18c229c2f632b"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.191119 15227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:24.192178 15242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.192396 15227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:24.192550 15227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "a3c286beca79475cad9273b326a75741"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3757635c940de7658760f7e3f00adb13"
server_key_iv: "156a6d723dbaf66daff18c229c2f632b"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.192658 15227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:24.210873 15227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:24.213289 15227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:24.213469 15227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:24.217610 15227 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:46857
I20260620 08:09:24.217669 15294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:46857 every 8 connection(s)
I20260620 08:09:24.218027 15227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:24.218891 15295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:24.221203 15295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741: Bootstrap starting.
I20260620 08:09:24.221526 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15227
I20260620 08:09:24.221663 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:24.221845 12819 external_mini_cluster.cc:1428] Setting key 1d7d4976be27cd4fad4addc9da20f139
I20260620 08:09:24.221844 15295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:24.222144 15295 log.cc:824] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:24.222858 15295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741: No bootstrap required, opened a new log
I20260620 08:09:24.225095 15295 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } }
I20260620 08:09:24.225268 15295 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:24.225304 15295 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c286beca79475cad9273b326a75741, State: Initialized, Role: FOLLOWER
I20260620 08:09:24.225421 15295 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } }
I20260620 08:09:24.225500 15295 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:24.225529 15295 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:24.225567 15295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:24.226231 15295 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } }
I20260620 08:09:24.226316 15295 leader_election.cc:302] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c286beca79475cad9273b326a75741; no voters:
I20260620 08:09:24.226481 15295 leader_election.cc:288] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:24.226581 15300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:24.226776 15295 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:24.226816 15300 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [term 1 LEADER]: Becoming Leader. State: Replica: a3c286beca79475cad9273b326a75741, State: Running, Role: LEADER
I20260620 08:09:24.226935 15300 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } }
I20260620 08:09:24.227406 15300 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c286beca79475cad9273b326a75741. Latest consensus state: current_term: 1 leader_uuid: "a3c286beca79475cad9273b326a75741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } } }
I20260620 08:09:24.227421 15301 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c286beca79475cad9273b326a75741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c286beca79475cad9273b326a75741" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46857 } } }
I20260620 08:09:24.227515 15300 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:24.227545 15301 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:24.228353 15312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:24.228894 15312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:24.230397 15312 catalog_manager.cc:1347] Generated new cluster ID: f1e61bdd75b04297b03c5f77b4a75e06
I20260620 08:09:24.230446 15312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:24.234476 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.222356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.12.132.254:46857)
0620 08:09:24.222528 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:24.222537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:24.222581 (+ 44us) server_negotiation.cc:407] Connection header received
0620 08:09:24.223093 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:24.223100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:24.223146 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:24.223263 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:24.224050 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.225017 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.226143 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.227031 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.227614 (+ 583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.230429 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:24.230433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:24.230434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:24.230446 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:24.232416 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.232868 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.232872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.232872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.232982 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.233193 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.233194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.233194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.233496 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:24.233651 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:24.234116 (+ 465us) server_negotiation.cc:299] Negotiation successful
0620 08:09:24.234258 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260620 08:09:24.246172 15312 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:24.246847 15312 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:24.251636 15312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3c286beca79475cad9273b326a75741: Generated new TSK 0
I20260620 08:09:24.251850 15312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:24.281136 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:24.363929 15321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:24.364132 15321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:24.364154 15321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:24.365453 15321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:24.365494 15321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:24.365520 15321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:24.366947 15321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:24.367185 15321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:24.367440 15321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:24.372644 15326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.372691 15321 server_base.cc:1034] running on GCE node
W20260620 08:09:24.372727 15329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.372821 15327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.373193 15321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:24.373401 15321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:24.374594 15321 hybrid_clock.cc:648] HybridClock initialized: now 1781942964374567 us; error 43 us; skew 500 ppm
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:24.377617 15321 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:24.378100 15321 webserver.cc:460] Webserver started at http://127.12.132.193:33077/ using document root <none> and password file <none>
I20260620 08:09:24.378325 15321 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:24.378391 15321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:24.378537 15321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:24.379572 15321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "8fcfdcf5df95407492eabb6d2a75cce2"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "59e1310971b973d5f1b575971b5378b9"
server_key_iv: "71919e3421b3e55973a804d7e820ce48"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.379931 15321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "8fcfdcf5df95407492eabb6d2a75cce2"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "59e1310971b973d5f1b575971b5378b9"
server_key_iv: "71919e3421b3e55973a804d7e820ce48"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.381618 15321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:24.382854 15336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.383080 15321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:24.383190 15321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "8fcfdcf5df95407492eabb6d2a75cce2"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "59e1310971b973d5f1b575971b5378b9"
server_key_iv: "71919e3421b3e55973a804d7e820ce48"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.383280 15321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:24.409615 15321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:24.411940 15321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:24.412111 15321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:24.412346 15321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:24.412711 15321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:24.412770 15321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.412814 15321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:24.412840 15321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.419034 15321 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:41039
I20260620 08:09:24.419075 15450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:41039 every 8 connection(s)
I20260620 08:09:24.419374 15321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:24.425976 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15321
I20260620 08:09:24.426093 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:24.426273 12819 external_mini_cluster.cc:1428] Setting key 73cb1b235b9359ffdb9f5fbd31795293
I20260620 08:09:24.430205 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.420019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:34915 (local address 127.12.132.254:46857)
0620 08:09:24.420062 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:24.420064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:24.420274 (+ 210us) server_negotiation.cc:407] Connection header received
0620 08:09:24.420877 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:24.420878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:24.420889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:24.420965 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:24.421861 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.422374 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.423336 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.423575 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.423659 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.426053 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:24.426055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:24.426055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:24.426062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:24.428204 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.428867 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.428869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.428870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.428944 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.429229 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.429230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.429231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.429413 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:24.429480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:24.430042 (+ 562us) server_negotiation.cc:299] Negotiation successful
0620 08:09:24.430137 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:24.430460 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.420001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46857 (local address 127.12.132.193:34915)
0620 08:09:24.420143 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:24.420176 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:24.420768 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:24.421037 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:24.421040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:24.421583 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:24.421764 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.421768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.422471 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.422472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.423234 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.423236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.423671 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.423672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.423789 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.424382 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:24.424391 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:24.425958 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:24.428314 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.428329 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.428333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.428761 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.429016 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.429017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.429017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.429153 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.429480 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:24.429489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:24.429828 (+ 339us) client_negotiation.cc:769] Sending connection context
0620 08:09:24.429980 (+ 152us) client_negotiation.cc:240] Negotiation successful
0620 08:09:24.430172 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260620 08:09:24.431008 15451 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46857
I20260620 08:09:24.431102 15451 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:24.431293 15451 heartbeater.cc:511] Master 127.12.132.254:46857 requested a full tablet report, sending...
I20260620 08:09:24.431789 15259 ts_manager.cc:194] Registered new tserver with Master: 8fcfdcf5df95407492eabb6d2a75cce2 (127.12.132.193:41039)
I20260620 08:09:24.432403 15259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:34915
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:24.457659 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:24.536733 15457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:24.536931 15457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:24.536953 15457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:24.538092 15457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:24.538136 15457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:24.538163 15457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:24.539577 15457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:24.539804 15457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:24.540031 15457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:24.544993 15463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.545068 15462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.545076 15465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.545208 15457 server_base.cc:1034] running on GCE node
I20260620 08:09:24.545385 15457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:24.545573 15457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:24.546770 15457 hybrid_clock.cc:648] HybridClock initialized: now 1781942964546744 us; error 30 us; skew 500 ppm
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:24.549624 15457 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:24.550055 15457 webserver.cc:460] Webserver started at http://127.12.132.194:42137/ using document root <none> and password file <none>
I20260620 08:09:24.550295 15457 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:24.550347 15457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:24.550486 15457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:24.551499 15457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "0a46a02b6ab54659a0fb1f775305259a"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "2cab76625fb62e7db88c229432b04f71"
server_key_iv: "2a1396f0fdf224c764621efc3e7341bc"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.551970 15457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "0a46a02b6ab54659a0fb1f775305259a"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "2cab76625fb62e7db88c229432b04f71"
server_key_iv: "2a1396f0fdf224c764621efc3e7341bc"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.553391 15457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:09:24.554314 15472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.554574 15457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:24.554675 15457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "0a46a02b6ab54659a0fb1f775305259a"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "2cab76625fb62e7db88c229432b04f71"
server_key_iv: "2a1396f0fdf224c764621efc3e7341bc"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.554769 15457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:24.563859 15457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:24.566123 15457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:24.566283 15457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:24.566514 15457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:24.566803 15457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:24.566859 15457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.566905 15457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:24.566933 15457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.572898 15457 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:34963
I20260620 08:09:24.572933 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:34963 every 8 connection(s)
I20260620 08:09:24.573233 15457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:24.582687 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15457
I20260620 08:09:24.582907 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:24.583073 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.574208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:55571 (local address 127.12.132.254:46857)
0620 08:09:24.574304 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:24.574307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:24.574666 (+ 359us) server_negotiation.cc:407] Connection header received
0620 08:09:24.575165 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:24.575166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:24.575179 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:24.575255 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:24.576197 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.576812 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.577817 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.577995 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.578051 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.580154 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:24.580155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:24.580155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:24.580161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:24.581525 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.581912 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.581913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.581914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.581970 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.582316 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.582317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.582317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.582418 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:24.582482 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:24.582876 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 08:09:24.582976 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 08:09:24.583108 12819 external_mini_cluster.cc:1428] Setting key 06815c48759c045792a608be189a655b
I20260620 08:09:24.583091 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.574206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46857 (local address 127.12.132.194:55571)
0620 08:09:24.574517 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:24.574556 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:24.575025 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:24.575325 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:24.575329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:24.575875 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:24.576072 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.576076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.576936 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.576937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.577726 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.577728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.578078 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.578079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.578195 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.578707 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:24.578715 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:24.580054 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:24.581601 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.581606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.581608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.581818 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.582122 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.582123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.582124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.582247 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.582502 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:24.582508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:24.582630 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 08:09:24.582726 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 08:09:24.582838 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":167,"threads_started":1}
I20260620 08:09:24.583652 15586 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46857
I20260620 08:09:24.583765 15586 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:24.583935 15586 heartbeater.cc:511] Master 127.12.132.254:46857 requested a full tablet report, sending...
I20260620 08:09:24.584333 15258 ts_manager.cc:194] Registered new tserver with Master: 0a46a02b6ab54659a0fb1f775305259a (127.12.132.194:34963)
I20260620 08:09:24.584779 15258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:55571
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:24.612149 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:24.697758 15591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:24.697971 15591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:24.697994 15591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:24.699141 15591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:24.699182 15591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:24.699209 15591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:24.700686 15591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:24.700937 15591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:24.701164 15591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:24.706243 15598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.706354 15600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:24.706295 15597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:24.706439 15591 server_base.cc:1034] running on GCE node
I20260620 08:09:24.706632 15591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:24.706883 15591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:24.708073 15591 hybrid_clock.cc:648] HybridClock initialized: now 1781942964708040 us; error 71 us; skew 500 ppm
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:24.710927 15591 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:24.711388 15591 webserver.cc:460] Webserver started at http://127.12.132.195:36611/ using document root <none> and password file <none>
I20260620 08:09:24.711622 15591 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:24.711691 15591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:24.711841 15591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:24.713011 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "e5e415fcafe64c4d85d347ec15d1d22c"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3c6fa176d6f7453ee34f828ee805ae13"
server_key_iv: "4dd9caedff32459705a02c7af7874ad6"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.713372 15591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "e5e415fcafe64c4d85d347ec15d1d22c"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3c6fa176d6f7453ee34f828ee805ae13"
server_key_iv: "4dd9caedff32459705a02c7af7874ad6"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.714725 15591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:24.715476 15607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.715693 15591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:24.715807 15591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "e5e415fcafe64c4d85d347ec15d1d22c"
format_stamp: "Formatted at 2026-06-20 08:09:24 on dist-test-slave-z0sp"
server_key: "3c6fa176d6f7453ee34f828ee805ae13"
server_key_iv: "4dd9caedff32459705a02c7af7874ad6"
server_key_version: "encryptionkey@0"
I20260620 08:09:24.715903 15591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:24.729372 15591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:24.731590 15591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:24.731746 15591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:24.731971 15591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:24.732290 15591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:24.732350 15591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.732390 15591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:24.732417 15591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:24.738550 15591 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:41111
I20260620 08:09:24.738585 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:41111 every 8 connection(s)
I20260620 08:09:24.738910 15591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942964, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:24.748402 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15591
I20260620 08:09:24.748471 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.739651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:38123 (local address 127.12.132.254:46857)
0620 08:09:24.739697 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:24.739700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:24.740172 (+ 472us) server_negotiation.cc:407] Connection header received
0620 08:09:24.740842 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:24.740843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:24.740853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:24.740912 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:24.741829 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.742331 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.743342 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.743538 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.743596 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.745871 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:24.745872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:24.745873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:24.745880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:24.747062 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.747484 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.747485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.747485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.747545 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:24.747816 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:24.747817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:24.747817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:24.747931 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:24.748008 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:24.748305 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 08:09:24.748380 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:24.748569 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:24.748770 12819 external_mini_cluster.cc:1428] Setting key 16458b5cfcdd6f14c965a8a4c22f8439
I20260620 08:09:24.748888 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:24.739642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46857 (local address 127.12.132.195:38123)
0620 08:09:24.739925 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:24.739990 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:24.740716 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:24.740995 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:24.740998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:24.741544 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:24.741728 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.741732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.742467 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.742468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.743235 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:24.743237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:24.743637 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:24.743638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:24.743749 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:24.744284 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:24.744300 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:24.745758 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:24.747170 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.747176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.747178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.747404 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.747617 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:24.747618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:24.747618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:24.747726 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:24.747999 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:24.748005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:24.748139 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 08:09:24.748274 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 08:09:24.748428 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260620 08:09:24.749639 15722 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46857
I20260620 08:09:24.749755 15722 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:24.749917 15722 heartbeater.cc:511] Master 127.12.132.254:46857 requested a full tablet report, sending...
I20260620 08:09:24.750299 15258 ts_manager.cc:194] Registered new tserver with Master: e5e415fcafe64c4d85d347ec15d1d22c (127.12.132.195:41111)
I20260620 08:09:24.750722 15258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:38123
I20260620 08:09:24.760807 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 08:09:24.772765 15299 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:24.771407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.12.132.254:46857)
0620 08:09:24.771484 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:24.771486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:24.771638 (+ 152us) server_negotiation.cc:407] Connection header received
0620 08:09:24.771688 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:24.771689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:24.771705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:24.771807 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:24.772682 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57798: BlockingRecv error: recv got EOF from 127.0.0.1:57798 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 08:09:24.773504 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15321
I20260620 08:09:24.778400 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15457
I20260620 08:09:24.782637 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15591
I20260620 08:09:24.786891 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15227
2026-06-20T08:09:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1878 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15735](info): set up 2 sockets
Jun 20 08:09:24 dist-test-slave-z0sp krb5kdc[15735](info): commencing operation
krb5kdc: starting...
W20260620 08:09:25.844997 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:25 dist-test-slave-z0sp krb5kdc[15735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:25.972843 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39155
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:26.060068 15748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:26.060272 15748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:26.060294 15748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:26.061475 15748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:26.061553 15748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:26.061580 15748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:26.061594 15748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:26.061606 15748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:26.063048 15748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:26.063235 15748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:26.063397 15748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:26.068130 15754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:26.068131 15753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:26.068396 15756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:26.068737 15748 server_base.cc:1034] running on GCE node
I20260620 08:09:26.068959 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:26.069182 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:26.070334 15748 hybrid_clock.cc:648] HybridClock initialized: now 1781942966070320 us; error 30 us; skew 500 ppm
Jun 20 08:09:26 dist-test-slave-z0sp krb5kdc[15735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:26.083343 15748 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:26.083832 15748 webserver.cc:460] Webserver started at http://127.12.132.254:37703/ using document root <none> and password file <none>
I20260620 08:09:26.084089 15748 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:26.084136 15748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:26.084262 15748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:26.085381 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "8fe292eb85d64b9c81d7394eeeec25bc"
format_stamp: "Formatted at 2026-06-20 08:09:26 on dist-test-slave-z0sp"
server_key: "7fa418684135d8752cd67bafdbb8ea14"
server_key_iv: "3ea8a8f5db6471464d2986845503596d"
server_key_version: "encryptionkey@0"
I20260620 08:09:26.085742 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "8fe292eb85d64b9c81d7394eeeec25bc"
format_stamp: "Formatted at 2026-06-20 08:09:26 on dist-test-slave-z0sp"
server_key: "7fa418684135d8752cd67bafdbb8ea14"
server_key_iv: "3ea8a8f5db6471464d2986845503596d"
server_key_version: "encryptionkey@0"
I20260620 08:09:26.087100 15748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:26.088127 15763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:26.088693 15748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:26.088815 15748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "8fe292eb85d64b9c81d7394eeeec25bc"
format_stamp: "Formatted at 2026-06-20 08:09:26 on dist-test-slave-z0sp"
server_key: "7fa418684135d8752cd67bafdbb8ea14"
server_key_iv: "3ea8a8f5db6471464d2986845503596d"
server_key_version: "encryptionkey@0"
I20260620 08:09:26.088898 15748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:26.113318 15748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:26.115579 15748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:26.115693 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:26.120350 15748 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:39155
I20260620 08:09:26.120417 15815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:39155 every 8 connection(s)
I20260620 08:09:26.120894 15748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:26.121574 15816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:26.123764 15816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc: Bootstrap starting.
I20260620 08:09:26.124311 15816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:26.124742 15816 log.cc:824] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:26.125365 15816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc: No bootstrap required, opened a new log
I20260620 08:09:26.126869 15816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } }
I20260620 08:09:26.127008 15816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:26.127030 15816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe292eb85d64b9c81d7394eeeec25bc, State: Initialized, Role: FOLLOWER
I20260620 08:09:26.127163 15816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } }
I20260620 08:09:26.127243 15816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:26.127266 15816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:26.127292 15816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:26.128120 15816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } }
I20260620 08:09:26.128198 15816 leader_election.cc:302] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fe292eb85d64b9c81d7394eeeec25bc; no voters:
I20260620 08:09:26.128307 15816 leader_election.cc:288] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:26.128400 15819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:26.128571 15819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [term 1 LEADER]: Becoming Leader. State: Replica: 8fe292eb85d64b9c81d7394eeeec25bc, State: Running, Role: LEADER
I20260620 08:09:26.128578 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15748
I20260620 08:09:26.128602 15816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:26.128675 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:26.128647 15819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } }
I20260620 08:09:26.128901 12819 external_mini_cluster.cc:1428] Setting key 558e32426b1ff25f06fc5185f192c03e
I20260620 08:09:26.128950 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fe292eb85d64b9c81d7394eeeec25bc. Latest consensus state: current_term: 1 leader_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } } }
I20260620 08:09:26.128962 15820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fe292eb85d64b9c81d7394eeeec25bc" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39155 } } }
I20260620 08:09:26.129031 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:26.129056 15820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:26.129402 15825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:26.129999 15825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:26.132048 15825 catalog_manager.cc:1347] Generated new cluster ID: 8bed1dc267b84235bedb583dd4e906da
I20260620 08:09:26.132117 15825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:26 dist-test-slave-z0sp krb5kdc[15735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:26.140404 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:26.129371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.12.132.254:39155)
0620 08:09:26.129675 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:26.129686 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:26.129723 (+ 37us) server_negotiation.cc:407] Connection header received
0620 08:09:26.129853 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:26.129860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:26.129922 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:26.130717 (+ 795us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:26.130854 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:26.131938 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:26.133049 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:26.133648 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:26.133764 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:26.136659 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:26.136663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:26.136663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:26.136675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:26.138637 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:26.139034 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:26.139037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:26.139037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:26.139145 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:26.139344 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:26.139345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:26.139345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:26.139667 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:26.139827 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:26.139943 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 08:09:26.140175 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
I20260620 08:09:26.141202 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:09:26.159759 15825 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:26.160475 15825 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:26.163003 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:26.159410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.12.132.254:39155)
0620 08:09:26.159450 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:26.159452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:26.159618 (+ 166us) server_negotiation.cc:407] Connection header received
0620 08:09:26.159741 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:26.159742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:26.159757 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:26.159823 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:26.160276 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:26.160837 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:26.161557 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:26.161751 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:26.161828 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:26.162517 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:26.162518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:26.162518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:26.162706 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:26.162720 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:26.162726 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:26.162796 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:26.162866 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 08:09:26.162904 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:26.171339 15825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fe292eb85d64b9c81d7394eeeec25bc: Generated new TSK 0
I20260620 08:09:26.171522 15825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:09:26.179661 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15748
2026-06-20T08:09:26Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1390 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:26 dist-test-slave-z0sp krb5kdc[15849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:26 dist-test-slave-z0sp krb5kdc[15849](info): set up 2 sockets
Jun 20 08:09:26 dist-test-slave-z0sp krb5kdc[15849](info): commencing operation
krb5kdc: starting...
W20260620 08:09:27.228833 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:27.357735 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37947
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:27.438798 15862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:27.439004 15862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:27.439029 15862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:27.440165 15862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:27.440367 15862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:27.440475 15862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:27.440542 15862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:27.440564 15862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:27.443359 15862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:27.443717 15862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:27.444016 15862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:27.449841 15870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.449853 15868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.449880 15862 server_base.cc:1034] running on GCE node
W20260620 08:09:27.449846 15867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.450332 15862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:27.450668 15862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:27.451889 15862 hybrid_clock.cc:648] HybridClock initialized: now 1781942967451887 us; error 44 us; skew 500 ppm
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:27.454694 15862 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:27.455402 15862 webserver.cc:460] Webserver started at http://127.12.132.254:38945/ using document root <none> and password file <none>
I20260620 08:09:27.455684 15862 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:27.455763 15862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:27.455932 15862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:27.457046 15862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "0e8698a7013648d38d95e4f0493425f9"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "0167f5609be0499126ca1c5a5fa80327"
server_key_iv: "59424bccc8e96e86f91a8d760dc40e67"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.457401 15862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "0e8698a7013648d38d95e4f0493425f9"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "0167f5609be0499126ca1c5a5fa80327"
server_key_iv: "59424bccc8e96e86f91a8d760dc40e67"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.458892 15862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:27.459801 15878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.460096 15862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:27.460227 15862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "0e8698a7013648d38d95e4f0493425f9"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "0167f5609be0499126ca1c5a5fa80327"
server_key_iv: "59424bccc8e96e86f91a8d760dc40e67"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.460301 15862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:27.478061 15862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:27.480386 15862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:27.480573 15862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:27.485105 15862 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:37947
I20260620 08:09:27.485184 15932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:37947 every 8 connection(s)
I20260620 08:09:27.485515 15862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:27.486515 15933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:27.489019 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: Bootstrap starting.
I20260620 08:09:27.489591 15933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:27.489876 15933 log.cc:824] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:27.490593 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: No bootstrap required, opened a new log
I20260620 08:09:27.492159 15933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } }
I20260620 08:09:27.492290 15933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:27.492312 15933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e8698a7013648d38d95e4f0493425f9, State: Initialized, Role: FOLLOWER
I20260620 08:09:27.492470 15933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } }
I20260620 08:09:27.492554 15933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:27.492578 15933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:27.492627 15933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:27.493113 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15862
I20260620 08:09:27.493237 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:27.493427 12819 external_mini_cluster.cc:1428] Setting key 2b4ddf4ab1ca63bb0ce036707582290d
I20260620 08:09:27.493554 15933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } }
I20260620 08:09:27.493685 15933 leader_election.cc:302] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e8698a7013648d38d95e4f0493425f9; no voters:
I20260620 08:09:27.493845 15933 leader_election.cc:288] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:27.494154 15933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:27.494278 15938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:27.494402 15938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0e8698a7013648d38d95e4f0493425f9, State: Running, Role: LEADER
I20260620 08:09:27.494482 15938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } }
I20260620 08:09:27.494918 15940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e8698a7013648d38d95e4f0493425f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } } }
I20260620 08:09:27.495026 15940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:27.494920 15936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e8698a7013648d38d95e4f0493425f9. Latest consensus state: current_term: 1 leader_uuid: "0e8698a7013648d38d95e4f0493425f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e8698a7013648d38d95e4f0493425f9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37947 } } }
I20260620 08:09:27.495289 15936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9 [sys.catalog]: This master's current role is: LEADER
W20260620 08:09:27.496023 15953 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:27.496093 15953 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:27.496155 15947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:27.496321 15947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:27.498445 15947 catalog_manager.cc:1347] Generated new cluster ID: c17e304008364baeb4791480936514df
I20260620 08:09:27.498515 15947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:27.506036 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:27.494039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.12.132.254:37947)
0620 08:09:27.494213 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:27.494223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:27.494948 (+ 725us) server_negotiation.cc:407] Connection header received
0620 08:09:27.495063 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:27.495070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:27.495131 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:27.496101 (+ 970us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:27.496203 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.497595 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.498418 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.499063 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.499177 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:27.501902 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:27.501906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:27.501906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:27.501917 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:27.504428 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.504840 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.504843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.504843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.504958 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.505171 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.505172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.505172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.505443 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:27.505583 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:27.505702 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 08:09:27.505828 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260620 08:09:27.517323 15947 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:27.517807 15947 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:27.528743 15947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e8698a7013648d38d95e4f0493425f9: Generated new TSK 0
I20260620 08:09:27.529134 15947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:27.555757 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:27.639555 15959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:27.639750 15959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:27.639773 15959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:27.640946 15959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:27.640985 15959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:27.641011 15959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:27.642343 15959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:27.642520 15959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:27.642686 15959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:09:27.648018 15959 server_base.cc:1034] running on GCE node
W20260620 08:09:27.647998 15967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.647997 15964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.648222 15965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.648576 15959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:27.649024 15959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:27.650204 15959 hybrid_clock.cc:648] HybridClock initialized: now 1781942967650180 us; error 43 us; skew 500 ppm
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:27.652969 15959 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:27.653614 15959 webserver.cc:460] Webserver started at http://127.12.132.193:42111/ using document root <none> and password file <none>
I20260620 08:09:27.653879 15959 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:27.653930 15959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:27.654074 15959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:27.655140 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "333a911d8bae4cfea22e002e505abfc5"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "08517a85b3289eff12e3a26dd016e8c0"
server_key_iv: "1cabdb75a0007877a492309a960611fb"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.655506 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "333a911d8bae4cfea22e002e505abfc5"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "08517a85b3289eff12e3a26dd016e8c0"
server_key_iv: "1cabdb75a0007877a492309a960611fb"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.656870 15959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:27.657677 15974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.657963 15959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:27.658035 15959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "333a911d8bae4cfea22e002e505abfc5"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "08517a85b3289eff12e3a26dd016e8c0"
server_key_iv: "1cabdb75a0007877a492309a960611fb"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.658118 15959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:27.672674 15959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:27.675310 15959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:27.675488 15959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:27.675769 15959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:27.676110 15959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:27.676165 15959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.676203 15959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:27.676230 15959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.682651 15959 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:46509
I20260620 08:09:27.682762 16088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:46509 every 8 connection(s)
I20260620 08:09:27.683054 15959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:27.691131 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 15959
I20260620 08:09:27.691267 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:27.691483 12819 external_mini_cluster.cc:1428] Setting key 227b50af9902b4d538c98847fa3cc2ea
I20260620 08:09:27.692762 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:27.683925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:58721 (local address 127.12.132.254:37947)
0620 08:09:27.684007 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:27.684010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:27.684049 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:09:27.684623 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:27.684626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:27.684644 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:27.684718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:27.685661 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.686169 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.687156 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.687358 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.687423 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:27.689827 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:27.689828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:27.689828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:27.689834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:27.691308 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.691790 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.691791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.691791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.691848 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.692098 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.692099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.692100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.692208 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:27.692278 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:27.692594 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 08:09:27.692672 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 08:09:27.692970 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:27.683766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37947 (local address 127.12.132.193:58721)
0620 08:09:27.683929 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:27.683970 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:27.684452 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:27.684790 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:27.684794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:27.685349 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:27.685551 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:27.685556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.686268 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.686269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:27.687061 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:27.687064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.687456 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.687458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:27.687563 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:27.688296 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:27.688304 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:27.689741 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:27.691405 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:27.691411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:27.691413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:27.691694 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:27.691915 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:27.691916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:27.691916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:27.692028 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:27.692281 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:27.692288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:27.692421 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 08:09:27.692569 (+ 148us) client_negotiation.cc:240] Negotiation successful
0620 08:09:27.692725 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260620 08:09:27.693449 16089 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37947
I20260620 08:09:27.693552 16089 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:27.693764 16089 heartbeater.cc:511] Master 127.12.132.254:37947 requested a full tablet report, sending...
I20260620 08:09:27.694279 15895 ts_manager.cc:194] Registered new tserver with Master: 333a911d8bae4cfea22e002e505abfc5 (127.12.132.193:46509)
I20260620 08:09:27.694869 15895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:58721
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:27.720178 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:27.808601 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:27.808800 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:27.808823 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:27.809918 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:27.809954 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:27.809980 16094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:27.811354 16094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:27.811542 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:27.811708 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:27.816843 16099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.816836 16103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.816987 16100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.817559 16094 server_base.cc:1034] running on GCE node
I20260620 08:09:27.817785 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:27.818033 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:27.819190 16094 hybrid_clock.cc:648] HybridClock initialized: now 1781942967819180 us; error 30 us; skew 500 ppm
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:27.821723 16094 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:27.822155 16094 webserver.cc:460] Webserver started at http://127.12.132.194:41701/ using document root <none> and password file <none>
I20260620 08:09:27.822381 16094 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:27.822456 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:27.822605 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:27.823648 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "67e2726a412448c580f1ee21b8c185e6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "e7647eecb64e9104bcd57cce579825f8"
server_key_iv: "bc33f767e976763ef4312d3eeaa1f998"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.824011 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "67e2726a412448c580f1ee21b8c185e6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "e7647eecb64e9104bcd57cce579825f8"
server_key_iv: "bc33f767e976763ef4312d3eeaa1f998"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.825433 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:27.826190 16110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.826404 16094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:27.826530 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "67e2726a412448c580f1ee21b8c185e6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "e7647eecb64e9104bcd57cce579825f8"
server_key_iv: "bc33f767e976763ef4312d3eeaa1f998"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.826609 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:27.834985 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:27.837112 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:27.837268 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:27.837487 16094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:27.837781 16094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:27.837836 16094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.837875 16094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:27.837903 16094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.843521 16094 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:40661
I20260620 08:09:27.843593 16224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:40661 every 8 connection(s)
I20260620 08:09:27.843909 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:27.845316 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16094
I20260620 08:09:27.845467 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:27.845703 12819 external_mini_cluster.cc:1428] Setting key cd4e54c69c64bb2e96ff56e47db20fd2
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:27.854562 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:27.844752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:39225 (local address 127.12.132.254:37947)
0620 08:09:27.844858 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:27.844861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:27.844983 (+ 122us) server_negotiation.cc:407] Connection header received
0620 08:09:27.845480 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:27.845482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:27.845497 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:27.845560 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:27.846572 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.847090 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.848338 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.848640 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.848701 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:27.851162 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:27.851163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:27.851164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:27.851170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:27.852632 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.853170 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.853172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.853173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.853280 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:27.853643 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:27.853644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:27.853645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:27.853897 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:27.853982 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:27.854364 (+ 382us) server_negotiation.cc:299] Negotiation successful
0620 08:09:27.854481 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 08:09:27.854749 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:27.844672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37947 (local address 127.12.132.194:39225)
0620 08:09:27.844836 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:27.844876 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:27.845361 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:27.845627 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:27.845631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:27.846276 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:27.846470 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:27.846475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.847191 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.847192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:27.848202 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:27.848206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:27.848724 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:27.848725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:27.848879 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:27.849451 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:27.849460 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:27.851049 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:27.852709 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:27.852716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:27.852718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:27.853032 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:27.853389 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:27.853390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:27.853390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:27.853535 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:27.853997 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:27.854005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:27.854175 (+ 170us) client_negotiation.cc:769] Sending connection context
0620 08:09:27.854308 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 08:09:27.854499 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
I20260620 08:09:27.855435 16225 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37947
I20260620 08:09:27.855525 16225 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:27.855664 16225 heartbeater.cc:511] Master 127.12.132.254:37947 requested a full tablet report, sending...
I20260620 08:09:27.856142 15895 ts_manager.cc:194] Registered new tserver with Master: 67e2726a412448c580f1ee21b8c185e6 (127.12.132.194:40661)
I20260620 08:09:27.856570 15895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:39225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:27.872026 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:27.953760 16230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:27.953966 16230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:27.953989 16230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:27.955101 16230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:27.955140 16230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:27.955166 16230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:27.956578 16230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:27.956794 16230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:27.957022 16230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:27.962507 16238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:27.962611 16235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.962649 16230 server_base.cc:1034] running on GCE node
W20260620 08:09:27.962831 16236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:27.963055 16230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:27.963281 16230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:27.964432 16230 hybrid_clock.cc:648] HybridClock initialized: now 1781942967964420 us; error 28 us; skew 500 ppm
Jun 20 08:09:27 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:27.967362 16230 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:27.968000 16230 webserver.cc:460] Webserver started at http://127.12.132.195:33181/ using document root <none> and password file <none>
I20260620 08:09:27.968350 16230 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:27.968439 16230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:27.968618 16230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:27.969677 16230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "a5d2b0f5e05a49f2944bc97f1de28bc6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "7df73f02ba91b4c0ae06abec9b337631"
server_key_iv: "cd13bc9625267d82a37ea8183708f8e8"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.970034 16230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "a5d2b0f5e05a49f2944bc97f1de28bc6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "7df73f02ba91b4c0ae06abec9b337631"
server_key_iv: "cd13bc9625267d82a37ea8183708f8e8"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.971524 16230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:27.972388 16245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:27.972610 16230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:27.972714 16230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "a5d2b0f5e05a49f2944bc97f1de28bc6"
format_stamp: "Formatted at 2026-06-20 08:09:27 on dist-test-slave-z0sp"
server_key: "7df73f02ba91b4c0ae06abec9b337631"
server_key_iv: "cd13bc9625267d82a37ea8183708f8e8"
server_key_version: "encryptionkey@0"
I20260620 08:09:27.972805 16230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:28.001448 16230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:28.003661 16230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:28.003769 16230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:28.004036 16230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:28.004341 16230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:28.004395 16230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:28.004428 16230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:28.004454 16230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:28.010509 16230 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:34341
I20260620 08:09:28.010565 16358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:34341 every 8 connection(s)
I20260620 08:09:28.010910 16230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:28.017268 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16230
I20260620 08:09:28.017366 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:28.017588 12819 external_mini_cluster.cc:1428] Setting key 57dd152890bb9eea842c81c6b1195c1b
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[15849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:28.021785 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:28.011632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:46127 (local address 127.12.132.254:37947)
0620 08:09:28.011691 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:28.011693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:28.012001 (+ 308us) server_negotiation.cc:407] Connection header received
0620 08:09:28.012539 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:28.012541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:28.012554 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:28.012622 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:28.013531 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.014041 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.015128 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.015310 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.015373 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:28.018364 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:28.018366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:28.018366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:28.018374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:28.020107 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:28.020621 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:28.020622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:28.020623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:28.020677 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:28.020983 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:28.020983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:28.020984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:28.021091 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:28.021148 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:28.021647 (+ 499us) server_negotiation.cc:299] Negotiation successful
0620 08:09:28.021719 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 08:09:28.022199 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:28.011620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37947 (local address 127.12.132.195:46127)
0620 08:09:28.011859 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:28.011898 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:28.012393 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:28.012689 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:28.012693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:28.013246 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:28.013422 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:28.013426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.014154 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.014155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:28.014974 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:28.014977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.015410 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.015411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:28.015516 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:28.016057 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:28.016070 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:28.018221 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:28.020199 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:28.020209 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:28.020210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:28.020477 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:28.020777 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:28.020778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:28.020778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:28.020902 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:28.021231 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:28.021244 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:28.021449 (+ 205us) client_negotiation.cc:769] Sending connection context
0620 08:09:28.021605 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 08:09:28.021804 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260620 08:09:28.022928 16359 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37947
I20260620 08:09:28.023042 16359 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:28.023320 16359 heartbeater.cc:511] Master 127.12.132.254:37947 requested a full tablet report, sending...
I20260620 08:09:28.023813 15895 ts_manager.cc:194] Registered new tserver with Master: a5d2b0f5e05a49f2944bc97f1de28bc6 (127.12.132.195:34341)
I20260620 08:09:28.024219 15895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:46127
I20260620 08:09:28.029857 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[15849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942968, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[15849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942968, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:28.054623 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:28.046778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.12.132.254:37947)
0620 08:09:28.046838 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:28.046841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:28.046982 (+ 141us) server_negotiation.cc:407] Connection header received
0620 08:09:28.047054 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:28.047055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:28.047065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:28.047178 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:28.047874 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.048358 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.049374 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:28.049569 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:28.049638 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:28.052022 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:28.052024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:28.052024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:28.052034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:28.053561 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:28.053922 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:28.053923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:28.053923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:28.053984 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:28.054141 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:28.054141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:28.054141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:28.054279 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:28.054350 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:28.054450 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 08:09:28.054513 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 08:09:28.054908 15895 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46142
I20260620 08:09:28.055409 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15959
I20260620 08:09:28.060284 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16094
I20260620 08:09:28.064934 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16230
I20260620 08:09:28.070600 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 15862
2026-06-20T08:09:28Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1892 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[16371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[16371](info): set up 2 sockets
Jun 20 08:09:28 dist-test-slave-z0sp krb5kdc[16371](info): commencing operation
krb5kdc: starting...
W20260620 08:09:29.112887 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:29.237358 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44141
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:44141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:29.316468 16384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:29.316756 16384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:29.316793 16384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:29.317934 16384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:29.317998 16384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:29.318024 16384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:29.318050 16384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:29.318076 16384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:29.319537 16384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:44141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:29.319792 16384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:29.320009 16384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:29.324800 16392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.324812 16390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.324817 16389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.324892 16384 server_base.cc:1034] running on GCE node
I20260620 08:09:29.325244 16384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:29.325524 16384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:29.326702 16384 hybrid_clock.cc:648] HybridClock initialized: now 1781942969326647 us; error 85 us; skew 500 ppm
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:29.329810 16384 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:29.330296 16384 webserver.cc:460] Webserver started at http://127.12.132.254:39501/ using document root <none> and password file <none>
I20260620 08:09:29.330528 16384 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:29.330572 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:29.330698 16384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:29.331756 16384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "76d328195918484d827a9deaad047eb9"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "da4c5d9365d9f4e8d77334636da3eeb5"
server_key_iv: "12d1febdf4d334c410f723275e59eab0"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.332132 16384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "76d328195918484d827a9deaad047eb9"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "da4c5d9365d9f4e8d77334636da3eeb5"
server_key_iv: "12d1febdf4d334c410f723275e59eab0"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.333674 16384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:29.334934 16399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.335168 16384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:29.335286 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "76d328195918484d827a9deaad047eb9"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "da4c5d9365d9f4e8d77334636da3eeb5"
server_key_iv: "12d1febdf4d334c410f723275e59eab0"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.335371 16384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:29.348752 16384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:29.350862 16384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:29.351037 16384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:29.355139 16384 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:44141
I20260620 08:09:29.355283 16451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:44141 every 8 connection(s)
I20260620 08:09:29.355739 16384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:29.356417 16452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:29.358639 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9: Bootstrap starting.
I20260620 08:09:29.359160 16452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:29.359408 16452 log.cc:824] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:29.360057 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9: No bootstrap required, opened a new log
I20260620 08:09:29.361624 16452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } }
I20260620 08:09:29.361755 16452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:29.361778 16452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d328195918484d827a9deaad047eb9, State: Initialized, Role: FOLLOWER
I20260620 08:09:29.361910 16452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } }
I20260620 08:09:29.361984 16452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:29.362006 16452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:29.362033 16452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:29.362416 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16384
I20260620 08:09:29.362545 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:29.362772 12819 external_mini_cluster.cc:1428] Setting key f06677b94ff3dec2fd591e494789c49f
I20260620 08:09:29.363132 16452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } }
I20260620 08:09:29.363250 16452 leader_election.cc:302] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76d328195918484d827a9deaad047eb9; no voters:
I20260620 08:09:29.363444 16452 leader_election.cc:288] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:29.363555 16455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:29.363723 16452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:29.363993 16455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [term 1 LEADER]: Becoming Leader. State: Replica: 76d328195918484d827a9deaad047eb9, State: Running, Role: LEADER
I20260620 08:09:29.364194 16455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } }
I20260620 08:09:29.365077 16458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76d328195918484d827a9deaad047eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } } }
I20260620 08:09:29.365191 16458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:29.365581 16459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d328195918484d827a9deaad047eb9. Latest consensus state: current_term: 1 leader_uuid: "76d328195918484d827a9deaad047eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d328195918484d827a9deaad047eb9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44141 } } }
I20260620 08:09:29.365664 16459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:29.365640 16466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:29.366402 16466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:29.367888 16466 catalog_manager.cc:1347] Generated new cluster ID: 09305df0ebc24c0385d6d430a366d10b
I20260620 08:09:29.367941 16466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:29.374388 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.363450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.12.132.254:44141)
0620 08:09:29.363700 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.363708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.363800 (+ 92us) server_negotiation.cc:407] Connection header received
0620 08:09:29.363912 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.363920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.363973 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.364104 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.365080 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.366677 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.367571 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.368101 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.368230 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.370476 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.370480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.370480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.370490 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.372743 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.373145 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.373148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.373148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.373242 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.373457 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.373458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.373458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.373688 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.373867 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.373938 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.374187 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260620 08:09:29.393702 16466 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:29.394510 16466 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:29.398818 16466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76d328195918484d827a9deaad047eb9: Generated new TSK 0
I20260620 08:09:29.399015 16466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:29.432149 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:29.528301 16479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:29.528553 16479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:29.528587 16479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:29.529677 16479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:29.529714 16479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:29.529742 16479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:29.531217 16479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:29.531395 16479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:29.531558 16479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:29.536681 16485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.536682 16487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.536767 16479 server_base.cc:1034] running on GCE node
W20260620 08:09:29.536733 16484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.537151 16479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:29.537413 16479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:29.538563 16479 hybrid_clock.cc:648] HybridClock initialized: now 1781942969538552 us; error 33 us; skew 500 ppm
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:29.541575 16479 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:29.542047 16479 webserver.cc:460] Webserver started at http://127.12.132.193:45819/ using document root <none> and password file <none>
I20260620 08:09:29.542268 16479 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:29.542332 16479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:29.542491 16479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:29.543505 16479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "24d12ec55b444c26a1c21571992003ab"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "d84b55e70a7df9038d81cac371510285"
server_key_iv: "60d1169ffac5f17809865233bf623cae"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.543872 16479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "24d12ec55b444c26a1c21571992003ab"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "d84b55e70a7df9038d81cac371510285"
server_key_iv: "60d1169ffac5f17809865233bf623cae"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.545364 16479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:29.546279 16494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.546689 16479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:29.546825 16479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "24d12ec55b444c26a1c21571992003ab"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "d84b55e70a7df9038d81cac371510285"
server_key_iv: "60d1169ffac5f17809865233bf623cae"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.546911 16479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:29.561517 16479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:29.563565 16479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:29.563957 16479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:29.564416 16479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:29.564960 16479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:29.565027 16479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.565071 16479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:29.565125 16479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.573469 16479 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:42947
I20260620 08:09:29.573518 16607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:42947 every 8 connection(s)
I20260620 08:09:29.573935 16479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:29.578107 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16479
I20260620 08:09:29.578224 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:29.578399 12819 external_mini_cluster.cc:1428] Setting key f2617fcd2057d329a7abe0e95b7b28af
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:29.585584 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.575014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:50765 (local address 127.12.132.254:44141)
0620 08:09:29.575084 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.575085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.575334 (+ 249us) server_negotiation.cc:407] Connection header received
0620 08:09:29.576359 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.576360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.576372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.576438 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.577107 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.577600 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.579010 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.579239 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.579298 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.582400 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.582401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.582402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.582409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.583831 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.584404 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.584405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.584405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.584458 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.584793 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.584794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.584794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.584899 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.584967 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.585385 (+ 418us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.585499 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 08:09:29.585731 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.574909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44141 (local address 127.12.132.193:50765)
0620 08:09:29.575205 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:29.575241 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:29.576228 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:29.576548 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:29.576551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:29.576811 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:29.576997 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.577001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.577694 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.577694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.578864 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.578867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.579321 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.579322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.579476 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.579960 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:29.579970 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:29.582156 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:29.583918 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.583927 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.583929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.584314 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.584566 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.584567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.584567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.584717 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.584968 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:29.584975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:29.585157 (+ 182us) client_negotiation.cc:769] Sending connection context
0620 08:09:29.585282 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 08:09:29.585456 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":140,"threads_started":1}
I20260620 08:09:29.586500 16608 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44141
I20260620 08:09:29.586607 16608 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:29.586823 16608 heartbeater.cc:511] Master 127.12.132.254:44141 requested a full tablet report, sending...
I20260620 08:09:29.587335 16416 ts_manager.cc:194] Registered new tserver with Master: 24d12ec55b444c26a1c21571992003ab (127.12.132.193:42947)
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
I20260620 08:09:29.587998 16416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:50765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:29.609154 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:29.691864 16613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:29.692080 16613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:29.692104 16613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:29.693327 16613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:29.693372 16613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:29.693398 16613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:29.694901 16613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:29.695134 16613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:29.695379 16613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:29.700578 16618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.700665 16619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.700572 16621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.701000 16613 server_base.cc:1034] running on GCE node
I20260620 08:09:29.701211 16613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:29.701440 16613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:29.702711 16613 hybrid_clock.cc:648] HybridClock initialized: now 1781942969702669 us; error 37 us; skew 500 ppm
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:29.706037 16613 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:29.706503 16613 webserver.cc:460] Webserver started at http://127.12.132.194:34567/ using document root <none> and password file <none>
I20260620 08:09:29.706734 16613 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:29.706801 16613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:29.706951 16613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:29.707985 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "a1bc253204e94e8282274030b1694b44"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "910fd4de99a0c4390a412196b63fdd6a"
server_key_iv: "a10b5fdb2c2f08cc01c391a3a749bb6d"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.708350 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "a1bc253204e94e8282274030b1694b44"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "910fd4de99a0c4390a412196b63fdd6a"
server_key_iv: "a10b5fdb2c2f08cc01c391a3a749bb6d"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.709707 16613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:29.710631 16628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.710925 16613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:29.711041 16613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "a1bc253204e94e8282274030b1694b44"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "910fd4de99a0c4390a412196b63fdd6a"
server_key_iv: "a10b5fdb2c2f08cc01c391a3a749bb6d"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.711124 16613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:29.725795 16613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:29.728253 16613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:29.728423 16613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:29.728674 16613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:29.728979 16613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:29.729031 16613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.729071 16613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:29.729099 16613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.735436 16613 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:43663
I20260620 08:09:29.735519 16741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:43663 every 8 connection(s)
I20260620 08:09:29.735805 16613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:29.744561 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16613
I20260620 08:09:29.744701 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:29.744961 12819 external_mini_cluster.cc:1428] Setting key bb25fef4b38aee13206b0bbc9c15f740
I20260620 08:09:29.746125 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.736531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:48035 (local address 127.12.132.254:44141)
0620 08:09:29.736582 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.736584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.736781 (+ 197us) server_negotiation.cc:407] Connection header received
0620 08:09:29.737728 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.737729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.737740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.737786 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.738350 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.738950 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.739914 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.740111 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.740173 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.742903 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.742904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.742904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.742913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.744311 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.744934 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.744935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.744935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.744998 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.745365 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.745365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.745366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.745513 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.745586 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.745987 (+ 401us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.746058 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:09:29.746500 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.736455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44141 (local address 127.12.132.194:48035)
0620 08:09:29.736663 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:29.736697 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:29.737583 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:29.737837 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:29.737840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:29.738072 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:29.738249 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.738252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.739027 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.739027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.739792 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.739795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.740194 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.740194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.740289 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.740916 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:29.740929 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:29.742755 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:29.744422 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.744430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.744433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.744833 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.745107 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.745108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.745108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.745264 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.745628 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:29.745639 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:29.745840 (+ 201us) client_negotiation.cc:769] Sending connection context
0620 08:09:29.745958 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 08:09:29.746168 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
I20260620 08:09:29.747123 16742 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44141
I20260620 08:09:29.747220 16742 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:29.747411 16742 heartbeater.cc:511] Master 127.12.132.254:44141 requested a full tablet report, sending...
I20260620 08:09:29.747787 16416 ts_manager.cc:194] Registered new tserver with Master: a1bc253204e94e8282274030b1694b44 (127.12.132.194:43663)
I20260620 08:09:29.748229 16416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:48035
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:29.773487 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:29.855441 16747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:29.855643 16747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:29.855665 16747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:29.857168 16747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:29.857218 16747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:29.857246 16747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:29.858844 16747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:29.859058 16747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:29.859267 16747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:29.866030 16752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:29.866030 16753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.866158 16747 server_base.cc:1034] running on GCE node
W20260620 08:09:29.866060 16755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:29.866453 16747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:29.866679 16747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:29.867869 16747 hybrid_clock.cc:648] HybridClock initialized: now 1781942969867835 us; error 49 us; skew 500 ppm
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:29.871022 16747 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:29.871611 16747 webserver.cc:460] Webserver started at http://127.12.132.195:40381/ using document root <none> and password file <none>
I20260620 08:09:29.871917 16747 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:29.871973 16747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:29.872151 16747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:29.873667 16747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "b42386d88184446e96a6d411d923580a"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "26bd8fe7c0ee29bfa2287751628106f6"
server_key_iv: "a5e640a3ff7ff0575fc5fe52e6162dde"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.874306 16747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "b42386d88184446e96a6d411d923580a"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "26bd8fe7c0ee29bfa2287751628106f6"
server_key_iv: "a5e640a3ff7ff0575fc5fe52e6162dde"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.875799 16747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 08:09:29.876838 16762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.877100 16747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:29.877218 16747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "b42386d88184446e96a6d411d923580a"
format_stamp: "Formatted at 2026-06-20 08:09:29 on dist-test-slave-z0sp"
server_key: "26bd8fe7c0ee29bfa2287751628106f6"
server_key_iv: "a5e640a3ff7ff0575fc5fe52e6162dde"
server_key_version: "encryptionkey@0"
I20260620 08:09:29.877298 16747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:29.885813 16747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:29.888391 16747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:29.888633 16747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:29.888929 16747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:29.889313 16747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:29.889374 16747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.889421 16747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:29.889468 16747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:29.895602 16747 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:40511
I20260620 08:09:29.895675 16876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:40511 every 8 connection(s)
I20260620 08:09:29.896019 16747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:29.898603 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16747
I20260620 08:09:29.898743 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:29.898954 12819 external_mini_cluster.cc:1428] Setting key 0c97a5cdeac4039588025d7b48ab2cdc
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:29.906530 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.896932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:56443 (local address 127.12.132.254:44141)
0620 08:09:29.897020 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.897022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.897165 (+ 143us) server_negotiation.cc:407] Connection header received
0620 08:09:29.898133 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.898134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.898145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.898204 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.898861 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.899648 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.900714 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.900894 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.900953 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.903093 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.903094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.903095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.903100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.904755 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.905351 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.905353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.905353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.905424 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.905778 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.905780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.905780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.905913 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.905989 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.906390 (+ 401us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.906463 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 08:09:29.906826 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.896796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44141 (local address 127.12.132.195:56443)
0620 08:09:29.897023 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:29.897071 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:29.898016 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:29.898268 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:29.898272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:29.898529 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:29.898759 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.898764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.899742 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.899743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.900623 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:29.900625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.900968 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.900969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:29.901078 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.901624 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:29.901632 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:29.902988 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:29.904887 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.904898 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.904901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.905244 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.905530 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:29.905531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:29.905531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:29.905671 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:29.906017 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:29.906023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:29.906233 (+ 210us) client_negotiation.cc:769] Sending connection context
0620 08:09:29.906347 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 08:09:29.906507 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260620 08:09:29.907431 16877 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44141
I20260620 08:09:29.907547 16877 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:29.907799 16877 heartbeater.cc:511] Master 127.12.132.254:44141 requested a full tablet report, sending...
I20260620 08:09:29.908375 16416 ts_manager.cc:194] Registered new tserver with Master: b42386d88184446e96a6d411d923580a (127.12.132.195:40511)
I20260620 08:09:29.908813 16416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:56443
I20260620 08:09:29.910987 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:29.936622 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.927872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.12.132.193:42947)
0620 08:09:29.928036 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.928040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.928050 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:29.928075 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.928077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.928091 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.928158 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.928799 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.929516 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.930352 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.930656 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.930722 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.933221 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.933260 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.933260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.933269 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.935030 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.935576 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.935579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.935579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.935637 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.935950 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.935951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.935951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.936091 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.936217 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.936379 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.936445 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
Jun 20 08:09:29 dist-test-slave-z0sp krb5kdc[16371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:29.945953 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:29.938119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.12.132.254:44141)
0620 08:09:29.938185 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:29.938187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:29.938199 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:29.938242 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:29.938243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:29.938252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:29.938292 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:29.938778 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.939622 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.940340 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:29.940609 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:29.940670 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:29.942788 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:29.942791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:29.942791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:29.942802 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:29.944377 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.944919 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.944920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.944920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.944988 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:29.945288 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:29.945289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:29.945290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:29.945483 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:29.945586 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:29.945800 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 08:09:29.945887 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 08:09:29.946276 16416 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41886
I20260620 08:09:29.946866 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16479
I20260620 08:09:29.952035 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16613
I20260620 08:09:29.956286 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16747
I20260620 08:09:29.960465 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16384
2026-06-20T08:09:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1888 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:30 dist-test-slave-z0sp krb5kdc[16889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:30 dist-test-slave-z0sp krb5kdc[16889](info): set up 2 sockets
Jun 20 08:09:30 dist-test-slave-z0sp krb5kdc[16889](info): commencing operation
krb5kdc: starting...
W20260620 08:09:31.013051 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:31.140724 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36637
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:31.224839 16902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:31.225091 16902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:31.225136 16902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:31.226285 16902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:31.226351 16902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:31.226378 16902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:31.226409 16902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:31.226440 16902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:31.227934 16902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:31.228199 16902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:31.228416 16902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:31.233847 16908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:31.233912 16910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:31.233836 16907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.233919 16902 server_base.cc:1034] running on GCE node
I20260620 08:09:31.234300 16902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:31.234613 16902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:31.235927 16902 hybrid_clock.cc:648] HybridClock initialized: now 1781942971235898 us; error 30 us; skew 500 ppm
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:31.244374 16902 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:31.244809 16902 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:31.244884 16902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:31.245045 16902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:31.246110 16902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "fd5327bb5ca64bcbb38ef0dd927a607a"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "5bb8295849a50090f71a3cc476fdb659"
server_key_iv: "6c78f599b6330a24e53143f181c9a745"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.246481 16902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "fd5327bb5ca64bcbb38ef0dd927a607a"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "5bb8295849a50090f71a3cc476fdb659"
server_key_iv: "6c78f599b6330a24e53143f181c9a745"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.247980 16902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:31.248862 16916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.249071 16902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:31.249163 16902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "fd5327bb5ca64bcbb38ef0dd927a607a"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "5bb8295849a50090f71a3cc476fdb659"
server_key_iv: "6c78f599b6330a24e53143f181c9a745"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.249218 16902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:31.281652 16902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:31.283950 16902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:31.284065 16902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:31.288041 16902 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:36637
I20260620 08:09:31.288070 16968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:36637 every 8 connection(s)
I20260620 08:09:31.288350 16902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:31.289366 16969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:31.291671 16969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: Bootstrap starting.
I20260620 08:09:31.292248 16969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:31.292606 16969 log.cc:824] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:31.293267 16969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: No bootstrap required, opened a new log
I20260620 08:09:31.294813 16969 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } }
I20260620 08:09:31.294972 16969 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:31.295009 16969 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5327bb5ca64bcbb38ef0dd927a607a, State: Initialized, Role: FOLLOWER
I20260620 08:09:31.295166 16969 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } }
I20260620 08:09:31.295262 16969 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:31.295305 16969 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:31.295354 16969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:31.296129 16969 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } }
I20260620 08:09:31.296247 16969 leader_election.cc:302] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd5327bb5ca64bcbb38ef0dd927a607a; no voters:
I20260620 08:09:31.296452 16969 leader_election.cc:288] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:31.296509 16972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:31.296664 16972 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [term 1 LEADER]: Becoming Leader. State: Replica: fd5327bb5ca64bcbb38ef0dd927a607a, State: Running, Role: LEADER
I20260620 08:09:31.296653 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16902
I20260620 08:09:31.296748 16972 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } }
I20260620 08:09:31.296784 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:31.296809 16969 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:31.296959 12819 external_mini_cluster.cc:1428] Setting key 71920372638f2abadd3016ee5cd79c73
I20260620 08:09:31.297254 16973 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } } }
I20260620 08:09:31.297351 16973 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:31.297962 16982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:31.298437 16974 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd5327bb5ca64bcbb38ef0dd927a607a. Latest consensus state: current_term: 1 leader_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5327bb5ca64bcbb38ef0dd927a607a" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36637 } } }
I20260620 08:09:31.298586 16974 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:31.298645 16982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:31.300230 16982 catalog_manager.cc:1347] Generated new cluster ID: 7eba64e7ca7b4bb0a9fa92fd642f644d
I20260620 08:09:31.300283 16982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:31.307817 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.297885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.12.132.254:36637)
0620 08:09:31.298059 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.298067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.298101 (+ 34us) server_negotiation.cc:407] Connection header received
0620 08:09:31.298214 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.298221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.298276 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.298429 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:31.299205 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.300539 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.301431 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.301872 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.301993 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.304213 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:31.304217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:31.304218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:31.304229 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:31.306329 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.306691 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.306694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.306694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.306780 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.306986 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.306987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.306988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.307225 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:31.307384 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.307478 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.307616 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260620 08:09:31.316267 16982 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:31.316879 16982 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:31.325887 16982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: Generated new TSK 0
I20260620 08:09:31.326054 16982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:31.360176 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:31.446374 16995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:31.446583 16995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:31.446615 16995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:31.447764 16995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:31.447804 16995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:31.447832 16995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:31.449369 16995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:31.449569 16995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:31.449746 16995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:31.454792 17000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:31.454933 17001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.455008 16995 server_base.cc:1034] running on GCE node
W20260620 08:09:31.455197 17003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.455510 16995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:31.455890 16995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:31.457127 16995 hybrid_clock.cc:648] HybridClock initialized: now 1781942971457109 us; error 30 us; skew 500 ppm
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:31.460202 16995 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:31.460616 16995 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:31.460690 16995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:31.460840 16995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:31.461874 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "7906552b34d142179a7d6cb5be650bb1"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "3daf1cf69b23c7594277b9f8d6305b04"
server_key_iv: "400cbc31fa5eae3453063764aa8f82b2"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.462229 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "7906552b34d142179a7d6cb5be650bb1"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "3daf1cf69b23c7594277b9f8d6305b04"
server_key_iv: "400cbc31fa5eae3453063764aa8f82b2"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.463515 16995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:09:31.464418 17009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.464692 16995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:31.464794 16995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "7906552b34d142179a7d6cb5be650bb1"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "3daf1cf69b23c7594277b9f8d6305b04"
server_key_iv: "400cbc31fa5eae3453063764aa8f82b2"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.464907 16995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:31.478811 16995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:31.481057 16995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:31.481220 16995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:31.481444 16995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:31.481746 16995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:31.481801 16995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.481830 16995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:31.481858 16995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.488152 16995 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:38735
I20260620 08:09:31.488197 17122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:38735 every 8 connection(s)
I20260620 08:09:31.488464 16995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:31.495551 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 16995
I20260620 08:09:31.495800 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:31.496054 12819 external_mini_cluster.cc:1428] Setting key 178536dcb109ed73685d93d2fc1a712e
I20260620 08:09:31.498402 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.489218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:42457 (local address 127.12.132.254:36637)
0620 08:09:31.489266 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.489268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.489502 (+ 234us) server_negotiation.cc:407] Connection header received
0620 08:09:31.490001 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.490002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.490014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.490076 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:31.490999 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.491519 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.492606 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.492824 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.492911 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.495200 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:31.495202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:31.495202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:31.495210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:31.496876 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.497431 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.497432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.497432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.497481 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.497795 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.497795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.497796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.497907 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:31.497959 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.498275 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.498337 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:31.498672 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.489190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36637 (local address 127.12.132.193:42457)
0620 08:09:31.489373 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.489405 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.489894 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.490142 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.490145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.490402 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:31.490896 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.490901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.491614 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.491615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.492478 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.492480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.492921 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.492922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.493049 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.493537 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:31.493545 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:31.495097 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:31.496990 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.496995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.496997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.497348 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.497541 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.497542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.497542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.497726 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.497970 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:31.497976 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:31.498133 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.498244 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.498405 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260620 08:09:31.499141 17123 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36637
I20260620 08:09:31.499246 17123 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:31.499400 17123 heartbeater.cc:511] Master 127.12.132.254:36637 requested a full tablet report, sending...
I20260620 08:09:31.499861 16933 ts_manager.cc:194] Registered new tserver with Master: 7906552b34d142179a7d6cb5be650bb1 (127.12.132.193:38735)
I20260620 08:09:31.500458 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:42457
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:31.525156 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:31.605728 17128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:31.605927 17128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:31.605952 17128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:31.607041 17128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:31.607080 17128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:31.607111 17128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:31.608556 17128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:31.608724 17128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:31.608886 17128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:31.613982 17136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.614050 17128 server_base.cc:1034] running on GCE node
W20260620 08:09:31.613989 17133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:31.613991 17134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.614353 17128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:31.614586 17128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:31.615726 17128 hybrid_clock.cc:648] HybridClock initialized: now 1781942971615711 us; error 42 us; skew 500 ppm
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:31.619155 17128 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:31.619594 17128 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:31.619650 17128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:31.619853 17128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:31.621209 17128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "b96b790e7e57389ac9881ab474d754d5"
server_key_iv: "79d313f86bdfee45934fb07ac9aa9f4e"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.621628 17128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "b96b790e7e57389ac9881ab474d754d5"
server_key_iv: "79d313f86bdfee45934fb07ac9aa9f4e"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.623274 17128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:31.624178 17142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.624415 17128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:31.624538 17128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "b96b790e7e57389ac9881ab474d754d5"
server_key_iv: "79d313f86bdfee45934fb07ac9aa9f4e"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.624627 17128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:31.636071 17128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:31.638435 17128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:31.638543 17128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:31.638716 17128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:31.638978 17128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:31.639011 17128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.639031 17128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:31.639045 17128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.644538 17128 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:40355
I20260620 08:09:31.644596 17255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:40355 every 8 connection(s)
I20260620 08:09:31.644850 17128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:31.650437 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17128
I20260620 08:09:31.650564 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:31.650962 12819 external_mini_cluster.cc:1428] Setting key 93415324547d12b0e3a2309e5efd7eff
I20260620 08:09:31.655143 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.645571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:49277 (local address 127.12.132.254:36637)
0620 08:09:31.645670 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.645673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.645830 (+ 157us) server_negotiation.cc:407] Connection header received
0620 08:09:31.646351 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.646352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.646363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.646417 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:31.647320 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.647830 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.648842 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.649051 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.649138 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.651438 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:31.651440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:31.651441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:31.651451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:31.653614 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.654105 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.654106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.654107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.654198 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.654506 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.654507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.654507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.654614 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:31.654670 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.655001 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.655080 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 08:09:31.655397 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.645476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36637 (local address 127.12.132.194:49277)
0620 08:09:31.645697 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.645741 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.646241 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.646492 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.646496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.646746 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:31.647218 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.647222 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.647943 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.647943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.648748 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.648750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.649146 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.649147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.649267 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.649707 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:31.649715 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:31.651283 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:31.653796 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.653802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.653803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.654025 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.654293 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.654294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.654295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.654435 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.654693 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:31.654701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:31.654859 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.654975 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.655148 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":110,"threads_started":1}
I20260620 08:09:31.655851 17256 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36637
I20260620 08:09:31.655954 17256 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:31.656116 17256 heartbeater.cc:511] Master 127.12.132.254:36637 requested a full tablet report, sending...
I20260620 08:09:31.656527 16933 ts_manager.cc:194] Registered new tserver with Master: 2ac5f1ade85e4a76ada2bacd3aa7b381 (127.12.132.194:40355)
I20260620 08:09:31.657055 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:49277
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:31.679913 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:31.759064 17261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:31.759272 17261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:31.759296 17261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:31.760362 17261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:31.760401 17261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:31.760426 17261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:31.761909 17261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:31.762081 17261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:31.762243 17261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:31.767139 17269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.767261 17261 server_base.cc:1034] running on GCE node
W20260620 08:09:31.767143 17267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:31.767156 17266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:31.767624 17261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:31.767933 17261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:31.769133 17261 hybrid_clock.cc:648] HybridClock initialized: now 1781942971769121 us; error 27 us; skew 500 ppm
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:31.771926 17261 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:31.772285 17261 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:31.772334 17261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:31.772485 17261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:31.773574 17261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "1645444237bb45bd8493c0a805e0d867"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "778e5c0d4e03d3a7266e81e888d93f5d"
server_key_iv: "d31c67a332314ce3368a44a50c705121"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.774034 17261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "1645444237bb45bd8493c0a805e0d867"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "778e5c0d4e03d3a7266e81e888d93f5d"
server_key_iv: "d31c67a332314ce3368a44a50c705121"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.775506 17261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:31.776683 17275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.776886 17261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:31.776988 17261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "1645444237bb45bd8493c0a805e0d867"
format_stamp: "Formatted at 2026-06-20 08:09:31 on dist-test-slave-z0sp"
server_key: "778e5c0d4e03d3a7266e81e888d93f5d"
server_key_iv: "d31c67a332314ce3368a44a50c705121"
server_key_version: "encryptionkey@0"
I20260620 08:09:31.777077 17261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:31.798933 17261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:31.800966 17261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:31.801079 17261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:31.801249 17261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:31.801507 17261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:31.801542 17261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.801564 17261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:31.801577 17261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:31.807493 17261 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:43747
I20260620 08:09:31.807602 17388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:43747 every 8 connection(s)
I20260620 08:09:31.807824 17261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:31.814608 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17261
I20260620 08:09:31.814719 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:31.814921 12819 external_mini_cluster.cc:1428] Setting key 5da476276429f98d0c44abc2a2f31577
Jun 20 08:09:31 dist-test-slave-z0sp krb5kdc[16889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942971, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:31.818338 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.808798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:40139 (local address 127.12.132.254:36637)
0620 08:09:31.808874 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.808876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.809168 (+ 292us) server_negotiation.cc:407] Connection header received
0620 08:09:31.809645 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.809647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.809658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.809720 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:31.810681 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.811529 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.812633 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.812812 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.812885 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.815607 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:31.815608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:31.815609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:31.815616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:31.817099 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.817508 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.817508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.817509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.817556 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.817767 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.817767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.817767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.817860 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:31.817915 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.818208 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.818267 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 08:09:31.818517 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.808798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36637 (local address 127.12.132.195:40139)
0620 08:09:31.809021 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.809058 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.809512 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.809789 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.809793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.810026 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:31.810578 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.810585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.811621 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.811622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.812443 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.812446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.812904 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.812904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.813007 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.813493 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:31.813506 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:31.815483 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:31.817188 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.817193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.817195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.817434 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.817614 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:31.817615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:31.817615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:31.817714 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:31.817941 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:31.817947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:31.818082 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.818174 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.818291 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":120,"threads_started":1}
I20260620 08:09:31.818996 17389 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36637
I20260620 08:09:31.819096 17389 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:31.819240 17389 heartbeater.cc:511] Master 127.12.132.254:36637 requested a full tablet report, sending...
I20260620 08:09:31.819705 16933 ts_manager.cc:194] Registered new tserver with Master: 1645444237bb45bd8493c0a805e0d867 (127.12.132.195:43747)
I20260620 08:09:31.820179 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:40139
I20260620 08:09:31.827605 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:31.835683 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.829477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.12.132.254:36637)
0620 08:09:31.829573 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.829574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.829677 (+ 103us) server_negotiation.cc:407] Connection header received
0620 08:09:31.829699 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.829700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.829709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.829770 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:31.830342 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.831025 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.831750 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.831954 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.832045 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.833218 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:31.833219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:31.833219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:31.833226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:31.834610 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.834942 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.834943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.834943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.834993 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:31.835155 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:31.835155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:31.835156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:31.835285 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:31.835359 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.835508 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.835566 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 08:09:31.837221 16933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:31.838757 16933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:31.844894 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:43747 (local address 127.0.0.1:46016)
0620 08:09:31.841328 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.841372 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.841446 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.841765 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.841769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.841771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:31.841850 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.841852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.842828 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.842829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.843641 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.843643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.844591 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.844592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.844672 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.844685 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.844762 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.844820 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":131,"threads_started":1}
I20260620 08:09:31.844920 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.12.132.195:43747)
0620 08:09:31.841606 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.841618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.841626 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:31.841654 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.841656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.841668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.841756 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:31.841926 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.842735 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.843760 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.844458 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.844590 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.844644 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.844783 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.844822 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260620 08:09:31.845934 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:40355 (local address 127.0.0.1:52390)
0620 08:09:31.841915 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.841925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.841984 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.842201 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.842202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.842205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:31.842251 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.842252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.843720 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.843721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.844848 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.844851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.845681 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.845682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.845754 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.845758 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.845831 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.845867 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":80,"threads_started":1}
I20260620 08:09:31.845969 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.12.132.194:40355)
0620 08:09:31.841898 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.841904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.841977 (+ 73us) server_negotiation.cc:407] Connection header received
0620 08:09:31.842059 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.842063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.842088 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.842179 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:31.842335 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.843583 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.844946 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.845598 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.845674 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.845722 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.845846 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.845887 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260620 08:09:31.845970 17323 tablet_service.cc:1467] Processing CreateTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 (DEFAULT_TABLE table=test-table [id=760c3fa93831433397b273566ff714c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:31.846350 17323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353bdc353f7b480a939ffefdb990c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:31.846545 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:38735 (local address 127.0.0.1:52936)
0620 08:09:31.841625 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.841639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.841704 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.842694 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.842695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.842698 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:31.842753 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.842755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.844167 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.844168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.845353 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.845375 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.846295 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.846296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.846361 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.846366 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.846430 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.846479 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":19,"thread_start_us":106,"threads_started":1}
I20260620 08:09:31.846766 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.841280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.12.132.193:38735)
0620 08:09:31.842469 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.842474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.842484 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:31.842525 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.842527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.842543 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.842822 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:31.842913 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.844001 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.845483 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.846223 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.846295 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.846357 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.846488 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.846531 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":98,"threads_started":1}
I20260620 08:09:31.846957 17190 tablet_service.cc:1467] Processing CreateTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 (DEFAULT_TABLE table=test-table [id=760c3fa93831433397b273566ff714c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:31.847230 17190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353bdc353f7b480a939ffefdb990c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:31.847357 17057 tablet_service.cc:1467] Processing CreateTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 (DEFAULT_TABLE table=test-table [id=760c3fa93831433397b273566ff714c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:31.847652 17057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353bdc353f7b480a939ffefdb990c6e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:31.849529 17406 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Bootstrap starting.
I20260620 08:09:31.850179 17406 tablet_bootstrap.cc:654] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:31.850720 17408 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: Bootstrap starting.
I20260620 08:09:31.850824 17406 log.cc:824] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:31.851174 17407 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Bootstrap starting.
I20260620 08:09:31.851377 17408 tablet_bootstrap.cc:654] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:31.851761 17406 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: No bootstrap required, opened a new log
I20260620 08:09:31.851842 17406 ts_tablet_manager.cc:1397] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:31.851958 17407 tablet_bootstrap.cc:654] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:31.852119 17408 log.cc:824] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:31.852579 17407 log.cc:824] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:31.853101 17408 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: No bootstrap required, opened a new log
I20260620 08:09:31.853171 17408 ts_tablet_manager.cc:1397] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:09:31.853297 17407 tablet_bootstrap.cc:492] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: No bootstrap required, opened a new log
I20260620 08:09:31.853358 17407 ts_tablet_manager.cc:1397] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:09:31.853991 17406 raft_consensus.cc:348] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.854339 17406 raft_consensus.cc:374] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:31.854404 17406 raft_consensus.cc:729] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac5f1ade85e4a76ada2bacd3aa7b381, State: Initialized, Role: FOLLOWER
I20260620 08:09:31.854528 17406 consensus_queue.cc:260] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.854825 17406 ts_tablet_manager.cc:1428] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:31.854983 17256 heartbeater.cc:503] Master 127.12.132.254:36637 was elected leader, sending a full tablet report...
I20260620 08:09:31.855218 17407 raft_consensus.cc:348] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.855126 17408 raft_consensus.cc:348] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.855373 17408 raft_consensus.cc:374] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:31.855372 17407 raft_consensus.cc:374] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:31.855419 17407 raft_consensus.cc:729] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1645444237bb45bd8493c0a805e0d867, State: Initialized, Role: FOLLOWER
I20260620 08:09:31.855417 17408 raft_consensus.cc:729] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7906552b34d142179a7d6cb5be650bb1, State: Initialized, Role: FOLLOWER
I20260620 08:09:31.855526 17408 consensus_queue.cc:260] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.855526 17407 consensus_queue.cc:260] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.855782 17408 ts_tablet_manager.cc:1428] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:31.855782 17407 ts_tablet_manager.cc:1428] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:09:31.855799 17389 heartbeater.cc:503] Master 127.12.132.254:36637 was elected leader, sending a full tablet report...
I20260620 08:09:31.855917 17123 heartbeater.cc:503] Master 127.12.132.254:36637 was elected leader, sending a full tablet report...
I20260620 08:09:31.868853 17414 raft_consensus.cc:482] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:31.868937 17414 raft_consensus.cc:504] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.869347 17414 leader_election.cc:288] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7906552b34d142179a7d6cb5be650bb1 (127.12.132.193:38735), 2ac5f1ade85e4a76ada2bacd3aa7b381 (127.12.132.194:40355)
I20260620 08:09:31.872527 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.869573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:38735 (local address 127.12.132.195:38273)
0620 08:09:31.869679 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.869692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.869746 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.869883 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.869884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.869885 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:31.869983 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.869985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.870665 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.870666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.871482 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.871484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.872245 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.872246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.872330 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.872334 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.872389 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.872426 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260620 08:09:31.872550 17402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.869692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:38273 (local address 127.12.132.193:38735)
0620 08:09:31.869739 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.869741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.869750 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:31.869814 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.869815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.869823 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.869879 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:31.870072 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.870582 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.871577 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.872157 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.872237 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.872246 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.872408 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.872459 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:09:31.872709 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.869782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:50757 (local address 127.12.132.194:40355)
0620 08:09:31.869825 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.869827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.869968 (+ 141us) server_negotiation.cc:407] Connection header received
0620 08:09:31.869993 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.869994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.870004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.870056 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:31.870226 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.870718 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.871748 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.872261 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.872333 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.872342 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.872606 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.872653 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:09:31.872790 17077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353bdc353f7b480a939ffefdb990c6e6" candidate_uuid: "1645444237bb45bd8493c0a805e0d867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7906552b34d142179a7d6cb5be650bb1" is_pre_election: true
I20260620 08:09:31.872910 17077 raft_consensus.cc:2454] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1645444237bb45bd8493c0a805e0d867 in term 0.
I20260620 08:09:31.872936 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.869729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:40355 (local address 127.12.132.195:50757)
0620 08:09:31.869872 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.869881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:31.869929 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:31.870073 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:31.870074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:31.870075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:31.870135 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.870137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.870838 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.870839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.871646 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:31.871649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.872402 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.872402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:31.872460 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.872463 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:31.872809 (+ 346us) client_negotiation.cc:240] Negotiation successful
0620 08:09:31.872849 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260620 08:09:31.873103 17276 leader_election.cc:302] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1645444237bb45bd8493c0a805e0d867, 7906552b34d142179a7d6cb5be650bb1; no voters:
I20260620 08:09:31.873221 17414 raft_consensus.cc:2781] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:31.873282 17414 raft_consensus.cc:482] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:31.873310 17414 raft_consensus.cc:3037] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:31.873385 17210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353bdc353f7b480a939ffefdb990c6e6" candidate_uuid: "1645444237bb45bd8493c0a805e0d867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" is_pre_election: true
I20260620 08:09:31.873486 17210 raft_consensus.cc:2454] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1645444237bb45bd8493c0a805e0d867 in term 0.
I20260620 08:09:31.874053 17414 raft_consensus.cc:504] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.874173 17414 leader_election.cc:288] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [CANDIDATE]: Term 1 election: Requested vote from peers 7906552b34d142179a7d6cb5be650bb1 (127.12.132.193:38735), 2ac5f1ade85e4a76ada2bacd3aa7b381 (127.12.132.194:40355)
I20260620 08:09:31.874317 17210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353bdc353f7b480a939ffefdb990c6e6" candidate_uuid: "1645444237bb45bd8493c0a805e0d867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381"
I20260620 08:09:31.874413 17210 raft_consensus.cc:3037] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:31.874423 17077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353bdc353f7b480a939ffefdb990c6e6" candidate_uuid: "1645444237bb45bd8493c0a805e0d867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7906552b34d142179a7d6cb5be650bb1"
I20260620 08:09:31.874583 17077 raft_consensus.cc:3037] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:31.875167 17210 raft_consensus.cc:2454] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1645444237bb45bd8493c0a805e0d867 in term 1.
I20260620 08:09:31.875423 17276 leader_election.cc:302] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1645444237bb45bd8493c0a805e0d867, 2ac5f1ade85e4a76ada2bacd3aa7b381; no voters:
I20260620 08:09:31.875555 17414 raft_consensus.cc:2781] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:31.875808 17414 raft_consensus.cc:686] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 1 LEADER]: Becoming Leader. State: Replica: 1645444237bb45bd8493c0a805e0d867, State: Running, Role: LEADER
I20260620 08:09:31.875918 17077 raft_consensus.cc:2454] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1645444237bb45bd8493c0a805e0d867 in term 1.
I20260620 08:09:31.875921 17414 consensus_queue.cc:237] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } }
I20260620 08:09:31.876750 16932 catalog_manager.cc:5495] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1645444237bb45bd8493c0a805e0d867 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "1645444237bb45bd8493c0a805e0d867" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1645444237bb45bd8493c0a805e0d867" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 43747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:31.885100 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.882739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.12.132.195:43747)
0620 08:09:31.882792 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:31.882794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:31.882802 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:31.882829 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:31.882829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:31.882861 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:31.882925 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:31.883159 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.883592 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.884370 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:31.884631 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:31.884689 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:31.884865 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:31.884946 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:31.885004 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:31.885025 (+ 21us) server_negotiation.cc:299] Negotiation successful
0620 08:09:31.885040 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 08:09:31.887046 17077 raft_consensus.cc:1261] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 1 FOLLOWER]: Refusing update from remote peer 1645444237bb45bd8493c0a805e0d867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:31.887167 17210 raft_consensus.cc:1261] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 1 FOLLOWER]: Refusing update from remote peer 1645444237bb45bd8493c0a805e0d867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:31.887377 17416 consensus_queue.cc:1035] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7906552b34d142179a7d6cb5be650bb1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 38735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:31.887521 17414 consensus_queue.cc:1035] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ac5f1ade85e4a76ada2bacd3aa7b381" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:31.889760 17419 mvcc.cc:205] Tried to move back new op lower bound from 7298838412846297088 to 7298838412804497408. Current Snapshot: MvccSnapshot[applied={T|T < 7298838412846297088}]
I20260620 08:09:31.890663 17420 mvcc.cc:205] Tried to move back new op lower bound from 7298838412846297088 to 7298838412804497408. Current Snapshot: MvccSnapshot[applied={T|T < 7298838412846297088 or (T in {7298838412846297088})}]
I20260620 08:09:31.890913 17421 mvcc.cc:205] Tried to move back new op lower bound from 7298838412846297088 to 7298838412804497408. Current Snapshot: MvccSnapshot[applied={T|T < 7298838412846297088}]
I20260620 08:09:31.892251 16933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:31.892343 16933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:31.893123 16933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5327bb5ca64bcbb38ef0dd927a607a: Sending DeleteTablet for 3 replicas of tablet 353bdc353f7b480a939ffefdb990c6e6
I20260620 08:09:31.893446 17057 tablet_service.cc:1514] Processing DeleteTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:31 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:52936
I20260620 08:09:31.893494 17190 tablet_service.cc:1514] Processing DeleteTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:31 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:52390
I20260620 08:09:31.893664 17429 tablet_replica.cc:330] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1: stopping tablet replica
I20260620 08:09:31.893702 17430 tablet_replica.cc:330] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: stopping tablet replica
I20260620 08:09:31.893792 17429 raft_consensus.cc:2229] T 353bdc353f7b480a939ffefdb990c6e6 P 7906552b34d142179a7d6cb5be650bb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:31.893797 17430 raft_consensus.cc:2229] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:31.893908 17430 raft_consensus.cc:2258] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:31.893934 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16995
I20260620 08:09:31.894105 17323 tablet_service.cc:1514] Processing DeleteTablet for tablet 353bdc353f7b480a939ffefdb990c6e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:31 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:46016
I20260620 08:09:31.896047 17430 ts_tablet_manager.cc:1905] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:31.899623 17430 ts_tablet_manager.cc:1918] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:31.899735 17430 log.cc:1196] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/353bdc353f7b480a939ffefdb990c6e6
I20260620 08:09:31.900022 17430 ts_tablet_manager.cc:1939] T 353bdc353f7b480a939ffefdb990c6e6 P 2ac5f1ade85e4a76ada2bacd3aa7b381: Deleting consensus metadata
I20260620 08:09:31.900595 16917 catalog_manager.cc:4841] TS 2ac5f1ade85e4a76ada2bacd3aa7b381 (127.12.132.194:40355): tablet 353bdc353f7b480a939ffefdb990c6e6 (table test-table [id=760c3fa93831433397b273566ff714c5]) successfully deleted
I20260620 08:09:31.900841 17431 tablet_replica.cc:330] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: stopping tablet replica
I20260620 08:09:31.900977 17431 raft_consensus.cc:2229] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:31.901151 17431 raft_consensus.cc:2258] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 08:09:31.901208 16917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:38735 (error 108)
I20260620 08:09:31.901648 17431 ts_tablet_manager.cc:1905] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:31.901871 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17128
I20260620 08:09:31.903928 17431 ts_tablet_manager.cc:1918] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:31.904017 17431 log.cc:1196] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/353bdc353f7b480a939ffefdb990c6e6
I20260620 08:09:31.904285 17431 ts_tablet_manager.cc:1939] T 353bdc353f7b480a939ffefdb990c6e6 P 1645444237bb45bd8493c0a805e0d867: Deleting consensus metadata
I20260620 08:09:31.904886 16918 catalog_manager.cc:4841] TS 1645444237bb45bd8493c0a805e0d867 (127.12.132.195:43747): tablet 353bdc353f7b480a939ffefdb990c6e6 (table test-table [id=760c3fa93831433397b273566ff714c5]) successfully deleted
I20260620 08:09:31.905093 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:31.904987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:38735 (local address 127.0.0.1:52938)
0620 08:09:31.905031 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:31.905067 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:38735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 08:09:31.905223 16917 catalog_manager.cc:4584] TS 7906552b34d142179a7d6cb5be650bb1 (127.12.132.193:38735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 353bdc353f7b480a939ffefdb990c6e6: Network error: Client connection negotiation failed: client connection to 127.12.132.193:38735: connect: Connection refused (error 111)
I20260620 08:09:31.908818 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17261
I20260620 08:09:31.914232 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 16902
2026-06-20T08:09:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1953 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T08:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:31Z Disabled control of system clock
I20260620 08:09:31.941797 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38327
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 08:09:32.027601 17436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:32.027809 17436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:32.027832 17436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:32.029876 17436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:32.029927 17436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:32.029942 17436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:32.029955 17436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:32.029968 17436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:32.031556 17436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:32.031764 17436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:32.031994 17436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:32.037531 17442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.037532 17441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.037680 17436 server_base.cc:1034] running on GCE node
W20260620 08:09:32.037544 17444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.038205 17436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:32.038807 17436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:32.040109 17436 hybrid_clock.cc:648] HybridClock initialized: now 1781942972040044 us; error 51 us; skew 500 ppm
I20260620 08:09:32.042323 17436 webserver.cc:460] Webserver started at http://127.12.132.254:45313/ using document root <none> and password file <none>
I20260620 08:09:32.042644 17436 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:32.042721 17436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:32.042888 17436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:32.044142 17436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "607b73e5c7374cee893ddfb2e8454d81"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "bdac5961d239c103500f58c7c998eea2"
server_key_iv: "5ff534390744743e58e090a955dd2e07"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.044631 17436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "607b73e5c7374cee893ddfb2e8454d81"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "bdac5961d239c103500f58c7c998eea2"
server_key_iv: "5ff534390744743e58e090a955dd2e07"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.046283 17436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:32.047243 17450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.047490 17436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:32.047605 17436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "607b73e5c7374cee893ddfb2e8454d81"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "bdac5961d239c103500f58c7c998eea2"
server_key_iv: "5ff534390744743e58e090a955dd2e07"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.047701 17436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:32.051194 17436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:32.051468 17436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:32.051618 17436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:32.055765 17436 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38327
I20260620 08:09:32.055806 17502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38327 every 8 connection(s)
I20260620 08:09:32.056143 17436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:32.057021 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17436
I20260620 08:09:32.057148 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:32.057155 17503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:32.057337 12819 external_mini_cluster.cc:1428] Setting key 9786734bf813eb297a2572ede3b2c488
I20260620 08:09:32.061002 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.058007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.12.132.254:38327)
0620 08:09:32.058233 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.058243 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.058273 (+ 30us) server_negotiation.cc:407] Connection header received
0620 08:09:32.059156 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.059178 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.059241 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.059483 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.060332 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.060337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.060337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.060342 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.060409 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.060421 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.060526 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.060719 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.060795 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":121,"threads_started":1}
I20260620 08:09:32.061618 17503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: Bootstrap starting.
I20260620 08:09:32.062302 17503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:32.062587 17503 log.cc:824] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:32.063613 17503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: No bootstrap required, opened a new log
I20260620 08:09:32.065757 17503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } }
I20260620 08:09:32.065937 17503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:32.065964 17503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b73e5c7374cee893ddfb2e8454d81, State: Initialized, Role: FOLLOWER
I20260620 08:09:32.066128 17503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } }
I20260620 08:09:32.066241 17503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:32.066293 17503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:32.066354 17503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:32.067034 17503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } }
I20260620 08:09:32.067132 17503 leader_election.cc:302] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607b73e5c7374cee893ddfb2e8454d81; no voters:
I20260620 08:09:32.067379 17503 leader_election.cc:288] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:32.067734 17508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:32.067744 17503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:32.067922 17508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [term 1 LEADER]: Becoming Leader. State: Replica: 607b73e5c7374cee893ddfb2e8454d81, State: Running, Role: LEADER
I20260620 08:09:32.068030 17508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } }
I20260620 08:09:32.068619 17509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "607b73e5c7374cee893ddfb2e8454d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } } }
I20260620 08:09:32.068584 17510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607b73e5c7374cee893ddfb2e8454d81. Latest consensus state: current_term: 1 leader_uuid: "607b73e5c7374cee893ddfb2e8454d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607b73e5c7374cee893ddfb2e8454d81" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38327 } } }
I20260620 08:09:32.068758 17510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:32.068936 17509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:32.069061 17517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:32.069610 17517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:32.071246 17517 catalog_manager.cc:1347] Generated new cluster ID: 8da3dfd6b5224b8a9bc71f6b0d74d7a6
I20260620 08:09:32.071309 17517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:32.077102 17517 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:32.077260 17517 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:32.086412 17517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: Generated new TSK 0
I20260620 08:09:32.086575 17517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:09:32.088267 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--builtin_ntp_servers=127.12.132.212:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 08:09:32.178258 17527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:32.178448 17527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:32.178470 17527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:32.179623 17527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:32.179661 17527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:32.179687 17527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:32.181069 17527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:32.181300 17527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:32.181576 17527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:32.187248 17532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.187412 17533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.187444 17535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.187836 17527 server_base.cc:1034] running on GCE node
I20260620 08:09:32.188172 17527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:32.188534 17527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:32.189829 17527 hybrid_clock.cc:648] HybridClock initialized: now 1781942972189807 us; error 38 us; skew 500 ppm
I20260620 08:09:32.191247 17527 webserver.cc:460] Webserver started at http://127.12.132.193:45507/ using document root <none> and password file <none>
I20260620 08:09:32.191509 17527 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:32.191581 17527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:32.191727 17527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:32.192890 17527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "0ac1bae64fc14ea09906f4ea4a69659e"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "69de3a4b118547c8fbc5248582b1c7ae"
server_key_iv: "f8f085de17d080dc51ca2b41c357b8e6"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.193250 17527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "0ac1bae64fc14ea09906f4ea4a69659e"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "69de3a4b118547c8fbc5248582b1c7ae"
server_key_iv: "f8f085de17d080dc51ca2b41c357b8e6"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.194562 17527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:32.195353 17541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.195516 17527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:32.195606 17527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "0ac1bae64fc14ea09906f4ea4a69659e"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "69de3a4b118547c8fbc5248582b1c7ae"
server_key_iv: "f8f085de17d080dc51ca2b41c357b8e6"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.195688 17527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:32.198891 17527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:32.199159 17527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:32.199296 17527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:32.199498 17527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:32.199983 17527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:32.200187 17527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.200244 17527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:32.200277 17527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.208127 17527 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35077
I20260620 08:09:32.208217 17654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35077 every 8 connection(s)
I20260620 08:09:32.208812 17527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:32.212105 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.209665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:41359 (local address 127.12.132.254:38327)
0620 08:09:32.209729 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.209731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.209975 (+ 244us) server_negotiation.cc:407] Connection header received
0620 08:09:32.210457 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.210459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.210465 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.210543 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.211832 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.211833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.211834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.211838 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.211854 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.211859 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.211924 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.212002 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.212017 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:09:32.212266 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.209638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38327 (local address 127.12.132.193:41359)
0620 08:09:32.209843 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.209882 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.210345 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.210611 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.210615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.210623 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.211681 (+ 1058us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.211682 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.211709 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.211736 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.211926 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.211929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.211931 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.211997 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.212057 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260620 08:09:32.213677 17655 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38327
I20260620 08:09:32.213757 17655 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:32.213874 17655 heartbeater.cc:511] Master 127.12.132.254:38327 requested a full tablet report, sending...
I20260620 08:09:32.214560 17467 ts_manager.cc:194] Registered new tserver with Master: 0ac1bae64fc14ea09906f4ea4a69659e (127.12.132.193:35077)
I20260620 08:09:32.215418 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17527
I20260620 08:09:32.215546 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:32.215729 12819 external_mini_cluster.cc:1428] Setting key 43f410613baf6de2d1ef0eafa89bed84
I20260620 08:09:32.216810 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--builtin_ntp_servers=127.12.132.212:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 08:09:32.297611 17658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:32.297843 17658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:32.297878 17658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:32.299147 17658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:32.299192 17658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:32.299232 17658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:32.300547 17658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:32.300796 17658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:32.301074 17658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:32.306329 17663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.306522 17666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.306488 17664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.306502 17658 server_base.cc:1034] running on GCE node
I20260620 08:09:32.306824 17658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:32.307015 17658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:32.308161 17658 hybrid_clock.cc:648] HybridClock initialized: now 1781942972308148 us; error 28 us; skew 500 ppm
I20260620 08:09:32.309605 17658 webserver.cc:460] Webserver started at http://127.12.132.194:38539/ using document root <none> and password file <none>
I20260620 08:09:32.309783 17658 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:32.309837 17658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:32.309926 17658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:32.310905 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "ecd1c36b2be343048e1fd9f20407cdcd"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "ba3155a41432cef6e16581672b501d00"
server_key_iv: "a58f7f2db83b41ce3e6756bd51d53597"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.311198 17658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "ecd1c36b2be343048e1fd9f20407cdcd"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "ba3155a41432cef6e16581672b501d00"
server_key_iv: "a58f7f2db83b41ce3e6756bd51d53597"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.312608 17658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:32.313447 17672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.313642 17658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:09:32.313725 17658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "ecd1c36b2be343048e1fd9f20407cdcd"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "ba3155a41432cef6e16581672b501d00"
server_key_iv: "a58f7f2db83b41ce3e6756bd51d53597"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.313797 17658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:32.316982 17658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:32.317219 17658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:32.317355 17658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:32.317560 17658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:32.317889 17658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:32.317927 17658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.317948 17658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:32.317960 17658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.323318 17658 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:41147
I20260620 08:09:32.323374 17785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:41147 every 8 connection(s)
I20260620 08:09:32.323700 17658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:32.326645 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.324514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:38933 (local address 127.12.132.254:38327)
0620 08:09:32.324587 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.324589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.324757 (+ 168us) server_negotiation.cc:407] Connection header received
0620 08:09:32.325226 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.325227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.325233 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.325296 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.326407 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.326408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.326408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.326412 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.326425 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.326429 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.326475 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.326561 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.326576 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:09:32.326849 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.324391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38327 (local address 127.12.132.194:38933)
0620 08:09:32.324648 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.324677 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.325122 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.325372 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.325375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.325381 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.326253 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.326254 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.326278 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.326303 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.326473 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.326477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.326481 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.326572 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.326626 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":128,"threads_started":1}
I20260620 08:09:32.327281 17786 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38327
I20260620 08:09:32.327358 17786 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:32.327515 17786 heartbeater.cc:511] Master 127.12.132.254:38327 requested a full tablet report, sending...
I20260620 08:09:32.327898 17467 ts_manager.cc:194] Registered new tserver with Master: ecd1c36b2be343048e1fd9f20407cdcd (127.12.132.194:41147)
I20260620 08:09:32.332590 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17658
I20260620 08:09:32.332708 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:32.332898 12819 external_mini_cluster.cc:1428] Setting key 901b7f8e3e18e4dccb4fab4d017a372a
I20260620 08:09:32.333981 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--builtin_ntp_servers=127.12.132.212:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 08:09:32.414675 17789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:32.414860 17789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:32.414880 17789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:32.416003 17789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:32.416040 17789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:32.416141 17789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:32.417655 17789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:32.417866 17789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:32.418151 17789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:32.423228 17797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.423197 17795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.423205 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.423624 17789 server_base.cc:1034] running on GCE node
I20260620 08:09:32.423806 17789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:32.424149 17789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:32.425308 17789 hybrid_clock.cc:648] HybridClock initialized: now 1781942972425298 us; error 31 us; skew 500 ppm
I20260620 08:09:32.426502 17789 webserver.cc:460] Webserver started at http://127.12.132.195:35119/ using document root <none> and password file <none>
I20260620 08:09:32.426697 17789 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:32.426736 17789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:32.426822 17789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:32.427786 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "e11c0216449a4bfcb6562fae23e4215b"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "58d34c96d260307da20575328a6d2561"
server_key_iv: "ea538e02bbbc295edd606bf214d4f10c"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.428084 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "e11c0216449a4bfcb6562fae23e4215b"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "58d34c96d260307da20575328a6d2561"
server_key_iv: "ea538e02bbbc295edd606bf214d4f10c"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.429394 17789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:32.430312 17803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.430632 17789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:32.430727 17789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "e11c0216449a4bfcb6562fae23e4215b"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "58d34c96d260307da20575328a6d2561"
server_key_iv: "ea538e02bbbc295edd606bf214d4f10c"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.430776 17789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:32.434010 17789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:32.434208 17789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:32.434294 17789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:32.434501 17789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:32.434724 17789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:32.434752 17789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.434772 17789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:32.434784 17789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.441114 17789 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:45519
I20260620 08:09:32.441236 17916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:45519 every 8 connection(s)
I20260620 08:09:32.441512 17789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:32.444769 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.442307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:52697 (local address 127.12.132.254:38327)
0620 08:09:32.442371 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.442373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.442736 (+ 363us) server_negotiation.cc:407] Connection header received
0620 08:09:32.443257 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.443258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.443264 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.443324 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.444510 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.444512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.444512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.444517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.444531 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.444536 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.444584 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.444685 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.444701 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 08:09:32.444962 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.442374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38327 (local address 127.12.132.195:52697)
0620 08:09:32.442622 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.442654 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.443135 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.443393 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.443396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.443403 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.444307 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.444307 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.444333 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.444358 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.444595 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.444600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.444603 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.444697 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.444754 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":132,"threads_started":1}
I20260620 08:09:32.445351 17917 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38327
I20260620 08:09:32.445431 17917 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:32.445542 17917 heartbeater.cc:511] Master 127.12.132.254:38327 requested a full tablet report, sending...
I20260620 08:09:32.445883 17467 ts_manager.cc:194] Registered new tserver with Master: e11c0216449a4bfcb6562fae23e4215b (127.12.132.195:45519)
I20260620 08:09:32.448846 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17789
I20260620 08:09:32.448966 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:32.449178 12819 external_mini_cluster.cc:1428] Setting key 72f966bcf84a1a57882f5f18a0470f4b
I20260620 08:09:32.450625 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:32.453509 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.452451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.12.132.254:38327)
0620 08:09:32.452553 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.452555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.452663 (+ 108us) server_negotiation.cc:407] Connection header received
0620 08:09:32.452684 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.452684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.452689 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.452737 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.453318 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.453320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.453320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.453323 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.453336 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.453339 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.453392 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.453451 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.453462 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 08:09:32.455454 17467 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:32.457314 17467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:32.462093 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:45519 (local address 127.0.0.1:58378)
0620 08:09:32.460398 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.460452 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.460545 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.461046 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.461047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.461051 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.461583 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.461584 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.461605 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.461633 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.461886 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.461893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.461897 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.461986 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.462003 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260620 08:09:32.462108 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.12.132.195:45519)
0620 08:09:32.460847 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.460852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.460865 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:09:32.460898 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.460900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.460911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.461259 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.461730 (+ 471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.461736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.461737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.461741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.461781 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.461803 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.461891 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.462009 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.462025 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260620 08:09:32.462386 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.12.132.194:41147)
0620 08:09:32.460673 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.460677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.460688 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:32.460724 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.460725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.460732 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.460819 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.461727 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.461734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.461735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.461739 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.461775 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.461795 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.462120 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.462260 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.462278 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260620 08:09:32.462517 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35077 (local address 127.0.0.1:43550)
0620 08:09:32.461399 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.461411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.461471 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.461668 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.461670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.461673 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.462111 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.462112 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.462121 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.462135 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.462386 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.462387 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.462388 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.462443 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.462459 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":140,"threads_started":1}
I20260620 08:09:32.462662 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:41147 (local address 127.0.0.1:46598)
0620 08:09:32.460519 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.460535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.460600 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.460849 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.460854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.460861 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.461587 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.461588 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.461605 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.461636 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.462176 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.462177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.462178 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.462566 (+ 388us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.462584 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260620 08:09:32.462809 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.460884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.12.132.193:35077)
0620 08:09:32.461331 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.461336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.461527 (+ 191us) server_negotiation.cc:407] Connection header received
0620 08:09:32.461561 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.461563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.461573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.461658 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.462222 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.462229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.462229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.462233 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.462267 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.462288 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.462350 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.462672 (+ 322us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.462694 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":112,"threads_started":1}
I20260620 08:09:32.463094 17851 tablet_service.cc:1467] Processing CreateTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c (DEFAULT_TABLE table=test-table [id=80da026968594c4e8b4766c1ee658b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:32.463241 17720 tablet_service.cc:1467] Processing CreateTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c (DEFAULT_TABLE table=test-table [id=80da026968594c4e8b4766c1ee658b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:32.463490 17851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28198013ed0449f3ae6f4e3b07ef7f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:32.463514 17720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28198013ed0449f3ae6f4e3b07ef7f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:32.463637 17589 tablet_service.cc:1467] Processing CreateTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c (DEFAULT_TABLE table=test-table [id=80da026968594c4e8b4766c1ee658b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:32.463971 17589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28198013ed0449f3ae6f4e3b07ef7f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:32.466897 17931 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Bootstrap starting.
I20260620 08:09:32.467180 17933 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Bootstrap starting.
I20260620 08:09:32.467859 17931 tablet_bootstrap.cc:654] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:32.467968 17933 tablet_bootstrap.cc:654] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:32.468278 17931 log.cc:824] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:32.468312 17933 log.cc:824] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:32.468715 17932 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Bootstrap starting.
I20260620 08:09:32.469328 17932 tablet_bootstrap.cc:654] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:32.469583 17931 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: No bootstrap required, opened a new log
I20260620 08:09:32.469619 17932 log.cc:824] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:32.469673 17931 ts_tablet_manager.cc:1397] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 08:09:32.470105 17933 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: No bootstrap required, opened a new log
I20260620 08:09:32.470180 17933 ts_tablet_manager.cc:1397] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 08:09:32.470347 17932 tablet_bootstrap.cc:492] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: No bootstrap required, opened a new log
I20260620 08:09:32.470407 17932 ts_tablet_manager.cc:1397] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:32.471803 17931 raft_consensus.cc:348] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.471972 17931 raft_consensus.cc:374] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:32.472016 17931 raft_consensus.cc:729] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd1c36b2be343048e1fd9f20407cdcd, State: Initialized, Role: FOLLOWER
I20260620 08:09:32.472239 17931 consensus_queue.cc:260] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.472555 17931 ts_tablet_manager.cc:1428] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:09:32.472714 17786 heartbeater.cc:503] Master 127.12.132.254:38327 was elected leader, sending a full tablet report...
I20260620 08:09:32.472973 17932 raft_consensus.cc:348] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.472967 17933 raft_consensus.cc:348] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.473122 17933 raft_consensus.cc:374] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:32.473122 17932 raft_consensus.cc:374] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:32.473161 17932 raft_consensus.cc:729] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11c0216449a4bfcb6562fae23e4215b, State: Initialized, Role: FOLLOWER
I20260620 08:09:32.473161 17933 raft_consensus.cc:729] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac1bae64fc14ea09906f4ea4a69659e, State: Initialized, Role: FOLLOWER
I20260620 08:09:32.473285 17932 consensus_queue.cc:260] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.473285 17933 consensus_queue.cc:260] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.473527 17932 ts_tablet_manager.cc:1428] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:32.473527 17933 ts_tablet_manager.cc:1428] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260620 08:09:32.473558 17917 heartbeater.cc:503] Master 127.12.132.254:38327 was elected leader, sending a full tablet report...
I20260620 08:09:32.473664 17655 heartbeater.cc:503] Master 127.12.132.254:38327 was elected leader, sending a full tablet report...
W20260620 08:09:32.574452 17787 tablet.cc:2369] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:09:32.692159 17918 tablet.cc:2369] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:09:32.709769 17656 tablet.cc:2369] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:32.770551 17938 raft_consensus.cc:482] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:32.770692 17938 raft_consensus.cc:504] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.771164 17938 leader_election.cc:288] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e11c0216449a4bfcb6562fae23e4215b (127.12.132.195:45519), ecd1c36b2be343048e1fd9f20407cdcd (127.12.132.194:41147)
I20260620 08:09:32.773229 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.771566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:45821 (local address 127.12.132.195:45519)
0620 08:09:32.771601 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.771602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.771613 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:32.771676 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.771677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.771684 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.771751 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.772908 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.772910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.772910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.772914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.772931 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.772936 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.773001 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.773140 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.773155 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 08:09:32.773229 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.771500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:35663 (local address 127.12.132.194:41147)
0620 08:09:32.771552 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.771554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.771618 (+ 64us) server_negotiation.cc:407] Connection header received
0620 08:09:32.771693 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.771694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.771698 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.771751 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.772909 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.772910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.772910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.772914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.772931 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.772936 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.773009 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.773140 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.773155 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:09:32.773272 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.771422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:45519 (local address 127.12.132.193:45821)
0620 08:09:32.771556 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.771565 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.771612 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.771884 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.771893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.771895 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.772791 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.772791 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.772796 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.772810 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.773053 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.773055 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.773056 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.773131 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.773160 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":13,"thread_start_us":49,"threads_started":1}
I20260620 08:09:32.773272 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.771422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:41147 (local address 127.12.132.193:35663)
0620 08:09:32.771547 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.771559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:32.771615 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:32.771884 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:32.771886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:32.771893 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:32.772781 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:09:32.772782 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:32.772791 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:09:32.772809 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:32.773053 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:32.773055 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:32.773056 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 08:09:32.773129 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 08:09:32.773168 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":19,"thread_start_us":62,"threads_started":1}
I20260620 08:09:32.773658 17871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28198013ed0449f3ae6f4e3b07ef7f4c" candidate_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e11c0216449a4bfcb6562fae23e4215b" is_pre_election: true
I20260620 08:09:32.773810 17871 raft_consensus.cc:2454] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac1bae64fc14ea09906f4ea4a69659e in term 0.
I20260620 08:09:32.773922 17740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28198013ed0449f3ae6f4e3b07ef7f4c" candidate_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" is_pre_election: true
I20260620 08:09:32.774019 17543 leader_election.cc:302] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac1bae64fc14ea09906f4ea4a69659e, e11c0216449a4bfcb6562fae23e4215b; no voters:
I20260620 08:09:32.774144 17938 raft_consensus.cc:2781] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:32.774263 17938 raft_consensus.cc:482] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:32.774281 17938 raft_consensus.cc:3037] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:32.774138 17740 raft_consensus.cc:2454] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac1bae64fc14ea09906f4ea4a69659e in term 0.
I20260620 08:09:32.775063 17938 raft_consensus.cc:504] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.775256 17938 leader_election.cc:288] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [CANDIDATE]: Term 1 election: Requested vote from peers e11c0216449a4bfcb6562fae23e4215b (127.12.132.195:45519), ecd1c36b2be343048e1fd9f20407cdcd (127.12.132.194:41147)
I20260620 08:09:32.775339 17871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28198013ed0449f3ae6f4e3b07ef7f4c" candidate_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e11c0216449a4bfcb6562fae23e4215b"
I20260620 08:09:32.775405 17740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28198013ed0449f3ae6f4e3b07ef7f4c" candidate_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd1c36b2be343048e1fd9f20407cdcd"
I20260620 08:09:32.775420 17871 raft_consensus.cc:3037] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:32.775483 17740 raft_consensus.cc:3037] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:32.776135 17871 raft_consensus.cc:2454] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac1bae64fc14ea09906f4ea4a69659e in term 1.
I20260620 08:09:32.776211 17740 raft_consensus.cc:2454] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac1bae64fc14ea09906f4ea4a69659e in term 1.
I20260620 08:09:32.776304 17543 leader_election.cc:302] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac1bae64fc14ea09906f4ea4a69659e, e11c0216449a4bfcb6562fae23e4215b; no voters:
I20260620 08:09:32.776400 17938 raft_consensus.cc:2781] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:32.776576 17938 raft_consensus.cc:686] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [term 1 LEADER]: Becoming Leader. State: Replica: 0ac1bae64fc14ea09906f4ea4a69659e, State: Running, Role: LEADER
I20260620 08:09:32.776672 17938 consensus_queue.cc:237] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } }
I20260620 08:09:32.777433 17467 catalog_manager.cc:5495] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac1bae64fc14ea09906f4ea4a69659e (127.12.132.193). New cstate: current_term: 1 leader_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac1bae64fc14ea09906f4ea4a69659e" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:32.818730 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.817640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.12.132.193:35077)
0620 08:09:32.817700 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.817702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.817717 (+ 15us) server_negotiation.cc:407] Connection header received
0620 08:09:32.817753 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.817753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.817765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.817819 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.818506 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.818508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.818508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.818512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.818526 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.818530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.818585 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.818664 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.818675 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 08:09:32.821578 17871 raft_consensus.cc:1261] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 1 FOLLOWER]: Refusing update from remote peer 0ac1bae64fc14ea09906f4ea4a69659e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:32.821635 17740 raft_consensus.cc:1261] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 1 FOLLOWER]: Refusing update from remote peer 0ac1bae64fc14ea09906f4ea4a69659e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:32.822090 17942 consensus_queue.cc:1035] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e11c0216449a4bfcb6562fae23e4215b" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:32.822378 17938 consensus_queue.cc:1035] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd1c36b2be343048e1fd9f20407cdcd" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 41147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:32.824927 17945 mvcc.cc:205] Tried to move back new op lower bound from 7298838416673550336 to 7298838416493809664. Current Snapshot: MvccSnapshot[applied={T|T < 7298838416673550336}]
I20260620 08:09:32.827800 17466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:32.827911 17466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:32.829013 17466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607b73e5c7374cee893ddfb2e8454d81: Sending DeleteTablet for 3 replicas of tablet 28198013ed0449f3ae6f4e3b07ef7f4c
I20260620 08:09:32.829427 17720 tablet_service.cc:1514] Processing DeleteTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:32 UTC) from {username='slave'} at 127.0.0.1:46598
I20260620 08:09:32.829423 17851 tablet_service.cc:1514] Processing DeleteTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:32 UTC) from {username='slave'} at 127.0.0.1:58378
I20260620 08:09:32.829661 17589 tablet_service.cc:1514] Processing DeleteTablet for tablet 28198013ed0449f3ae6f4e3b07ef7f4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:32 UTC) from {username='slave'} at 127.0.0.1:43550
I20260620 08:09:32.829898 17957 tablet_replica.cc:330] T 28198013ed0449f3ae6f4e3b07ef7f4c P 0ac1bae64fc14ea09906f4ea4a69659e: stopping tablet replica
I20260620 08:09:32.829937 17956 tablet_replica.cc:330] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: stopping tablet replica
I20260620 08:09:32.830006 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17527
I20260620 08:09:32.830032 17956 raft_consensus.cc:2229] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:32.830134 17956 raft_consensus.cc:2258] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:32.831210 17956 ts_tablet_manager.cc:1905] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:32.831476 17955 tablet_replica.cc:330] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: stopping tablet replica
I20260620 08:09:32.831557 17955 raft_consensus.cc:2229] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:32.831632 17955 raft_consensus.cc:2258] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:32.832010 17955 ts_tablet_manager.cc:1905] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:32.833258 17956 ts_tablet_manager.cc:1918] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:32.833353 17956 log.cc:1196] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/28198013ed0449f3ae6f4e3b07ef7f4c
I20260620 08:09:32.833564 17955 ts_tablet_manager.cc:1918] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:32.833622 17956 ts_tablet_manager.cc:1939] T 28198013ed0449f3ae6f4e3b07ef7f4c P e11c0216449a4bfcb6562fae23e4215b: Deleting consensus metadata
I20260620 08:09:32.833644 17955 log.cc:1196] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/28198013ed0449f3ae6f4e3b07ef7f4c
I20260620 08:09:32.833959 17955 ts_tablet_manager.cc:1939] T 28198013ed0449f3ae6f4e3b07ef7f4c P ecd1c36b2be343048e1fd9f20407cdcd: Deleting consensus metadata
I20260620 08:09:32.834079 17452 catalog_manager.cc:4841] TS e11c0216449a4bfcb6562fae23e4215b (127.12.132.195:45519): tablet 28198013ed0449f3ae6f4e3b07ef7f4c (table test-table [id=80da026968594c4e8b4766c1ee658b13]) successfully deleted
I20260620 08:09:32.834314 17454 catalog_manager.cc:4841] TS ecd1c36b2be343048e1fd9f20407cdcd (127.12.132.194:41147): tablet 28198013ed0449f3ae6f4e3b07ef7f4c (table test-table [id=80da026968594c4e8b4766c1ee658b13]) successfully deleted
W20260620 08:09:32.836750 17451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:35077 (error 108)
I20260620 08:09:32.837007 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17658
I20260620 08:09:32.841503 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.841340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35077 (local address 127.0.0.1:43564)
0620 08:09:32.841413 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:32.841460 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 08:09:32.841624 17451 catalog_manager.cc:4584] TS 0ac1bae64fc14ea09906f4ea4a69659e (127.12.132.193:35077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28198013ed0449f3ae6f4e3b07ef7f4c: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35077: connect: Connection refused (error 111)
I20260620 08:09:32.842563 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17789
I20260620 08:09:32.847545 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17436
2026-06-20T08:09:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (932 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 08:09:32.867679 12819 mini_oidc.cc:152] Starting JWKS server
I20260620 08:09:32.868876 12819 webserver.cc:460] Webserver started at https://127.0.0.1:33777/ using document root <none> and password file <none>
I20260620 08:09:32.868994 12819 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 08:09:32.869184 12819 webserver.cc:460] Webserver started at http://127.0.0.1:43915/ using document root <none> and password file <none>
2026-06-20T08:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:32Z Disabled control of system clock
I20260620 08:09:32.877615 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:35055
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:35055
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33777/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/testchainca.cert with env {}
W20260620 08:09:32.956239 17964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:32.956422 17964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:32.956444 17964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:32.957588 17964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:32.957629 17964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:32.957641 17964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:32.957654 17964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:32.957666 17964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 08:09:32.957679 17964 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 08:09:32.957690 17964 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33777/jwks.json
I20260620 08:09:32.959163 17964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:35055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:35055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33777/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:32.959339 17964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:32.959494 17964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:32.964030 17972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.964039 17969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:32.964046 17970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:32.964272 17964 server_base.cc:1034] running on GCE node
I20260620 08:09:32.964589 17964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:32.964877 17964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:32.966053 17964 hybrid_clock.cc:648] HybridClock initialized: now 1781942972966059 us; error 65 us; skew 500 ppm
I20260620 08:09:32.967437 17964 webserver.cc:460] Webserver started at http://127.12.132.254:39285/ using document root <none> and password file <none>
I20260620 08:09:32.967691 17964 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:32.967761 17964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:32.967896 17964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:32.968961 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "b17d40615a2912ffdcfd9c4c09251bcf"
server_key_iv: "5ffc311400ee095b00081be67c02c2ca"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.969307 17964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "b17d40615a2912ffdcfd9c4c09251bcf"
server_key_iv: "5ffc311400ee095b00081be67c02c2ca"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.970734 17964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:32.971850 17978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:32.972136 17964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:32.972260 17964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2"
format_stamp: "Formatted at 2026-06-20 08:09:32 on dist-test-slave-z0sp"
server_key: "b17d40615a2912ffdcfd9c4c09251bcf"
server_key_iv: "5ffc311400ee095b00081be67c02c2ca"
server_key_version: "encryptionkey@0"
I20260620 08:09:32.972344 17964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:32.984750 17964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:32.985057 17964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:32.985211 17964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:32.989048 17964 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:35055
I20260620 08:09:32.989125 18030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:35055 every 8 connection(s)
I20260620 08:09:32.989454 17964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:32.990389 18031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:32.991865 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 17964
I20260620 08:09:32.991972 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:32.992164 12819 external_mini_cluster.cc:1428] Setting key 9b576a4b700338d5f6d7b666230f31e5
I20260620 08:09:32.993038 18031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2: Bootstrap starting.
I20260620 08:09:32.993606 18031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:32.993928 18031 log.cc:824] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:32.994694 18031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2: No bootstrap required, opened a new log
I20260620 08:09:32.997015 18031 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } }
I20260620 08:09:32.997148 18031 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:32.997169 18031 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a9a09a10ff42b3a7f89d17f21e7fc2, State: Initialized, Role: FOLLOWER
I20260620 08:09:32.997292 18031 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } }
I20260620 08:09:32.997375 18031 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:32.997407 18031 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:32.997463 18031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:32.998189 18031 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } }
I20260620 08:09:32.998226 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:32.992873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.12.132.254:35055)
0620 08:09:32.993053 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:32.993060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:32.993105 (+ 45us) server_negotiation.cc:407] Connection header received
0620 08:09:32.993531 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:32.993537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:32.993590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:32.993757 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:32.994046 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:32.995083 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:32.996176 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:32.996641 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:32.996739 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:32.997498 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:32.997502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:32.997502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:32.997696 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:32.997736 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:32.997750 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:32.997819 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:32.997921 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 08:09:32.998002 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260620 08:09:32.998303 18031 leader_election.cc:302] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a9a09a10ff42b3a7f89d17f21e7fc2; no voters:
I20260620 08:09:32.998449 18031 leader_election.cc:288] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:32.998548 18036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:32.998687 18036 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [term 1 LEADER]: Becoming Leader. State: Replica: b2a9a09a10ff42b3a7f89d17f21e7fc2, State: Running, Role: LEADER
I20260620 08:09:32.998709 18031 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:32.998769 18036 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } }
I20260620 08:09:32.999166 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:09:32.999147 18037 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } } }
I20260620 08:09:32.999228 18037 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:32.999133 18036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a9a09a10ff42b3a7f89d17f21e7fc2. Latest consensus state: current_term: 1 leader_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a9a09a10ff42b3a7f89d17f21e7fc2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35055 } } }
I20260620 08:09:32.999367 18036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:32.999464 18043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:33.000603 18043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:33.002315 18043 catalog_manager.cc:1347] Generated new cluster ID: e1604ac28d324ebfa61b9b479b7366dd
I20260620 08:09:33.002367 18043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:33.005877 18043 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:33.006806 18043 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:33.014772 18043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a9a09a10ff42b3a7f89d17f21e7fc2: Generated new TSK 0
I20260620 08:09:33.014955 18043 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 08:09:33.046212 18061 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 08:09:33.046321 18061 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 08:09:33.046511 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.023174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.12.132.254:35055)
0620 08:09:33.023231 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.023233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.023405 (+ 172us) server_negotiation.cc:407] Connection header received
0620 08:09:33.023505 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.023505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.023517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.023579 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0620 08:09:33.023726 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.024253 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.025446 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.025639 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.025702 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.025920 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 08:09:33.046197 (+ 20277us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 08:09:33.046403 (+ 206us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49390: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 08:09:33.046589 18034 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49390: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 08:09:33.046802 18060 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:33.023088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:35055 (local address 127.0.0.1:49390)
0620 08:09:33.023323 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:33.023334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:33.023410 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:33.023603 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:33.023604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:33.023606 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 08:09:33.023659 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.023661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.024334 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.024334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.025339 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.025342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.025732 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.025733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.025793 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.025815 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 08:09:33.046374 (+ 20559us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 08:09:33.046485 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.132.254:35055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.132.254:35055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 08:09:33.047315 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 17964
2026-06-20T08:09:33Z chronyd exiting
I20260620 08:09:33.083617 12819 test_util.cc:175] -----------------------------------------------
I20260620 08:09:33.083709 12819 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781942949664683-12819-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 08:09:33.084414 12819 mini_oidc.cc:152] Starting JWKS server
I20260620 08:09:33.085150 12819 webserver.cc:460] Webserver started at https://127.0.0.1:42203/ using document root <none> and password file <none>
I20260620 08:09:33.085220 12819 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 08:09:33.085388 12819 webserver.cc:460] Webserver started at http://127.0.0.1:41087/ using document root <none> and password file <none>
2026-06-20T08:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:33Z Disabled control of system clock
I20260620 08:09:33.094592 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45261
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:45261
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42203/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/testchainca.cert with env {}
W20260620 08:09:33.180480 18067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:33.180701 18067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:33.180723 18067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:33.181842 18067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:33.181878 18067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:33.181892 18067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:33.181905 18067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:33.181917 18067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 08:09:33.181929 18067 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 08:09:33.181941 18067 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42203/jwks.json
I20260620 08:09:33.183350 18067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:45261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42203/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:33.183530 18067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:33.183678 18067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:33.189038 18075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:33.189038 18072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:33.189157 18073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:33.189559 18067 server_base.cc:1034] running on GCE node
I20260620 08:09:33.189827 18067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:33.190145 18067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:33.191357 18067 hybrid_clock.cc:648] HybridClock initialized: now 1781942973191345 us; error 48 us; skew 500 ppm
I20260620 08:09:33.193073 18067 webserver.cc:460] Webserver started at http://127.12.132.254:41565/ using document root <none> and password file <none>
I20260620 08:09:33.193365 18067 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:33.193445 18067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:33.193612 18067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:33.194876 18067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "6a3fbd1e5f724182b6869c62d9f93780"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "41815c0b7be6c99e380451fcde24c84a"
server_key_iv: "d9d8c617fa2a8cd30af4fc3b741b5418"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.195369 18067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "6a3fbd1e5f724182b6869c62d9f93780"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "41815c0b7be6c99e380451fcde24c84a"
server_key_iv: "d9d8c617fa2a8cd30af4fc3b741b5418"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.196812 18067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260620 08:09:33.197621 18081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:33.197825 18067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:33.197929 18067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "6a3fbd1e5f724182b6869c62d9f93780"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "41815c0b7be6c99e380451fcde24c84a"
server_key_iv: "d9d8c617fa2a8cd30af4fc3b741b5418"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.198025 18067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:33.221297 18067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:33.221604 18067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:33.221762 18067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:33.225813 18067 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:45261
I20260620 08:09:33.225878 18133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:45261 every 8 connection(s)
I20260620 08:09:33.226234 18067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:33.227084 18134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:33.229588 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18067
I20260620 08:09:33.229737 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:33.229985 12819 external_mini_cluster.cc:1428] Setting key 6bab762151cce3b4122e7bd6f40ee260
I20260620 08:09:33.230999 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780: Bootstrap starting.
I20260620 08:09:33.231900 18134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:33.232240 18134 log.cc:824] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:33.233135 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780: No bootstrap required, opened a new log
I20260620 08:09:33.235400 18134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } }
I20260620 08:09:33.235594 18134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:33.235632 18134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3fbd1e5f724182b6869c62d9f93780, State: Initialized, Role: FOLLOWER
I20260620 08:09:33.235782 18134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } }
I20260620 08:09:33.235862 18134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:33.235885 18134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:33.235962 18134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:33.236672 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.230712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.12.132.254:45261)
0620 08:09:33.230955 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.230966 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.231006 (+ 40us) server_negotiation.cc:407] Connection header received
0620 08:09:33.231896 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.231907 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.231989 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.232156 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:33.232375 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.233664 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.234438 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.235118 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.235248 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.236035 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:33.236039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:33.236039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:33.236211 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:33.236242 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:33.236252 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:33.236331 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:33.236414 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 08:09:33.236519 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260620 08:09:33.236683 18134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } }
I20260620 08:09:33.236832 18134 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3fbd1e5f724182b6869c62d9f93780; no voters:
I20260620 08:09:33.236982 18134 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:33.237020 18139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:33.237123 18139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [term 1 LEADER]: Becoming Leader. State: Replica: 6a3fbd1e5f724182b6869c62d9f93780, State: Running, Role: LEADER
I20260620 08:09:33.237224 18134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:33.237207 18139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } }
I20260620 08:09:33.237547 18139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a3fbd1e5f724182b6869c62d9f93780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } } }
I20260620 08:09:33.237623 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:09:33.237628 18139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:33.237744 18139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3fbd1e5f724182b6869c62d9f93780. Latest consensus state: current_term: 1 leader_uuid: "6a3fbd1e5f724182b6869c62d9f93780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3fbd1e5f724182b6869c62d9f93780" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45261 } } }
I20260620 08:09:33.237810 18139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:33.238301 18145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:33.238907 18145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:33.240828 18145 catalog_manager.cc:1347] Generated new cluster ID: 599bb263f1b742ae8fef24fde228c36e
I20260620 08:09:33.240948 18145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:33.255767 18145 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:33.256433 18145 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:33.259161 18145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a3fbd1e5f724182b6869c62d9f93780: Generated new TSK 0
I20260620 08:09:33.259454 18145 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 08:09:33.285207 18164 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 08:09:33.285307 18164 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260620 08:09:33.285894 18163 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:33.270407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45261 (local address 127.0.0.1:49982)
0620 08:09:33.270571 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:33.270582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:33.270653 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:33.270821 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:33.270822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:33.270824 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 08:09:33.270880 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.270882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.271915 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.271916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.272660 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.272662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.273143 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.273144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.273187 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.273192 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 08:09:33.285687 (+ 12495us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 08:09:33.285813 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.132.254:45261: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
I20260620 08:09:33.285894 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.270504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49982 (local address 127.12.132.254:45261)
0620 08:09:33.270552 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.270554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.270669 (+ 115us) server_negotiation.cc:407] Connection header received
0620 08:09:33.270704 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.270705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.270715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.270802 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0620 08:09:33.270981 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.271814 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.272739 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.273060 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.273135 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.273283 (+ 148us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 08:09:33.285505 (+ 12222us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 08:09:33.285774 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 08:09:33.285970 18137 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260620 08:09:33.286332 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18067
2026-06-20T08:09:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 08:09:33.325006 12819 mini_oidc.cc:152] Starting JWKS server
I20260620 08:09:33.325690 12819 webserver.cc:460] Webserver started at https://127.0.0.1:45597/ using document root <none> and password file <none>
I20260620 08:09:33.325754 12819 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 08:09:33.325935 12819 webserver.cc:460] Webserver started at http://127.0.0.1:39313/ using document root <none> and password file <none>
2026-06-20T08:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:33Z Disabled control of system clock
I20260620 08:09:33.336112 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37287
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37287
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 08:09:33.426682 18170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:33.426970 18170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:33.427011 18170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:33.428265 18170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:33.428328 18170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:33.428350 18170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:33.428382 18170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:33.428408 18170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 08:09:33.428435 18170 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 08:09:33.428457 18170 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45597/jwks.json
I20260620 08:09:33.430809 18170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:33.431190 18170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:33.431526 18170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:33.436475 18175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:33.436429 18178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:33.436662 18170 server_base.cc:1034] running on GCE node
W20260620 08:09:33.436424 18176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:33.437002 18170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:33.437273 18170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:33.438441 18170 hybrid_clock.cc:648] HybridClock initialized: now 1781942973438431 us; error 31 us; skew 500 ppm
I20260620 08:09:33.439803 18170 webserver.cc:460] Webserver started at http://127.12.132.254:42951/ using document root <none> and password file <none>
I20260620 08:09:33.440058 18170 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:33.440107 18170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:33.440237 18170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:33.441396 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "419ba25aa78648f9a654945e07760688"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "42b336c3e58eee9995c39acf94d5fb80"
server_key_iv: "e7493da2b741ef32ddcf99b17579d608"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.441773 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "419ba25aa78648f9a654945e07760688"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "42b336c3e58eee9995c39acf94d5fb80"
server_key_iv: "e7493da2b741ef32ddcf99b17579d608"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.443287 18170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:33.444401 18184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:33.444654 18170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:33.444756 18170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "419ba25aa78648f9a654945e07760688"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "42b336c3e58eee9995c39acf94d5fb80"
server_key_iv: "e7493da2b741ef32ddcf99b17579d608"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.444810 18170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:33.461094 18170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:33.461632 18170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:33.461791 18170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:33.466776 18170 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:37287
I20260620 08:09:33.466809 18236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:37287 every 8 connection(s)
I20260620 08:09:33.467160 18170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:33.468060 18237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:33.470877 18237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: Bootstrap starting.
I20260620 08:09:33.471414 18237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:33.471645 18237 log.cc:824] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:33.472249 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18170
I20260620 08:09:33.472275 18237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: No bootstrap required, opened a new log
I20260620 08:09:33.472359 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:33.472615 12819 external_mini_cluster.cc:1428] Setting key 68991ce9cfa4c4b3bfe9b0e5beffd1aa
I20260620 08:09:33.474201 18237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } }
I20260620 08:09:33.474356 18237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:33.474412 18237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419ba25aa78648f9a654945e07760688, State: Initialized, Role: FOLLOWER
I20260620 08:09:33.474539 18237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } }
I20260620 08:09:33.474645 18237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:33.474689 18237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:33.474737 18237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:33.475435 18237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } }
I20260620 08:09:33.475564 18237 leader_election.cc:302] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 419ba25aa78648f9a654945e07760688; no voters:
I20260620 08:09:33.475731 18237 leader_election.cc:288] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:33.475771 18242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:33.475899 18242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [term 1 LEADER]: Becoming Leader. State: Replica: 419ba25aa78648f9a654945e07760688, State: Running, Role: LEADER
I20260620 08:09:33.475987 18237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:33.475996 18242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } }
I20260620 08:09:33.476315 18242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 419ba25aa78648f9a654945e07760688. Latest consensus state: current_term: 1 leader_uuid: "419ba25aa78648f9a654945e07760688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } } }
I20260620 08:09:33.476405 18242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:33.476325 18243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "419ba25aa78648f9a654945e07760688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419ba25aa78648f9a654945e07760688" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37287 } } }
I20260620 08:09:33.476637 18243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688 [sys.catalog]: This master's current role is: LEADER
W20260620 08:09:33.477782 18258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:33.477838 18258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:33.477882 18248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:33.478052 18248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:33.479483 18248 catalog_manager.cc:1347] Generated new cluster ID: 8c57ec5f882a484b976f9c94208300f9
I20260620 08:09:33.479537 18248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:33.480185 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.473135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.12.132.254:37287)
0620 08:09:33.473321 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.473328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.473351 (+ 23us) server_negotiation.cc:407] Connection header received
0620 08:09:33.474192 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.474199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.474266 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.474414 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:33.474631 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.476312 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.477425 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.477979 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.478131 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.479068 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:33.479071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:33.479071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:33.479290 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:33.479330 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:33.479343 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:33.479647 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:33.479684 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 08:09:33.479825 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260620 08:09:33.481393 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:09:33.515182 18248 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:33.515712 18248 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:33.518481 18248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 419ba25aa78648f9a654945e07760688: Generated new TSK 0
I20260620 08:09:33.518666 18248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:09:33.531208 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.526767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.12.132.254:37287)
0620 08:09:33.526849 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.526852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.526869 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:09:33.526929 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.526930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.526947 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.527078 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0620 08:09:33.527222 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.528086 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.528962 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.529283 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.529366 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.529537 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 08:09:33.530954 (+ 1417us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 08:09:33.531120 (+ 166us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 08:09:33.531294 18241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37528: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 08:09:33.533128 18266 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:33.526614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37287 (local address 127.0.0.1:37528)
0620 08:09:33.526779 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:33.526790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:33.526848 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:33.527084 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:33.527085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:33.527086 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0620 08:09:33.527138 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.527140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.528190 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.528191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.528871 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.528874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.529373 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.529374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.529418 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.529422 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 08:09:33.531081 (+ 1659us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 08:09:33.533030 (+ 1949us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.132.254:37287: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":1757,"thread_start_us":93,"threads_started":1}
I20260620 08:09:33.533319 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18170
E20260620 08:09:33.544037 18267 webserver.cc:533] Webserver: sslize failed:
E20260620 08:09:33.544245 18267 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T08:09:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T08:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:33Z Disabled control of system clock
I20260620 08:09:33.579995 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39683
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:39683
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/testchainca.cert with env {}
W20260620 08:09:33.661432 18271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:33.661803 18271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:33.661859 18271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:33.664098 18271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:33.664214 18271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:33.664242 18271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:33.664264 18271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:33.664296 18271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 08:09:33.664322 18271 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 08:09:33.664352 18271 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 08:09:33.666255 18271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:39683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:33.666553 18271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:33.666766 18271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:33.672008 18279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:33.672123 18271 server_base.cc:1034] running on GCE node
W20260620 08:09:33.671996 18277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:33.672019 18276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:33.672695 18271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:33.673231 18271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:33.674396 18271 hybrid_clock.cc:648] HybridClock initialized: now 1781942973674374 us; error 44 us; skew 500 ppm
I20260620 08:09:33.675952 18271 webserver.cc:460] Webserver started at http://127.12.132.254:38721/ using document root <none> and password file <none>
I20260620 08:09:33.676195 18271 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:33.676241 18271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:33.676369 18271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:33.677465 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "76cbfd0dfb924d26bfb24de278f5275f"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "35ec5bb963f273c8b2fd3c80b22d5a9b"
server_key_iv: "bc4e26ffce5fcc3e304e198ea007f261"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.677855 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "76cbfd0dfb924d26bfb24de278f5275f"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "35ec5bb963f273c8b2fd3c80b22d5a9b"
server_key_iv: "bc4e26ffce5fcc3e304e198ea007f261"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.679198 18271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:33.679976 18285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:33.680377 18271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:33.680459 18271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "76cbfd0dfb924d26bfb24de278f5275f"
format_stamp: "Formatted at 2026-06-20 08:09:33 on dist-test-slave-z0sp"
server_key: "35ec5bb963f273c8b2fd3c80b22d5a9b"
server_key_iv: "bc4e26ffce5fcc3e304e198ea007f261"
server_key_version: "encryptionkey@0"
I20260620 08:09:33.680588 18271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:33.694154 18271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:33.694468 18271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:33.694576 18271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:33.698520 18271 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:39683
I20260620 08:09:33.698549 18337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:39683 every 8 connection(s)
I20260620 08:09:33.698875 18271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:33.699810 18338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:33.701995 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f: Bootstrap starting.
I20260620 08:09:33.702551 18338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:33.702849 18338 log.cc:824] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:33.703540 18338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f: No bootstrap required, opened a new log
I20260620 08:09:33.704667 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18271
I20260620 08:09:33.704792 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:33.704980 12819 external_mini_cluster.cc:1428] Setting key 1fc6719349d859e298d716aa980770b1
I20260620 08:09:33.705459 18338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } }
I20260620 08:09:33.705646 18338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:33.705683 18338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cbfd0dfb924d26bfb24de278f5275f, State: Initialized, Role: FOLLOWER
I20260620 08:09:33.705807 18338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } }
I20260620 08:09:33.705879 18338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:33.705900 18338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:33.705977 18338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:33.706512 18338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } }
I20260620 08:09:33.706597 18338 leader_election.cc:302] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76cbfd0dfb924d26bfb24de278f5275f; no voters:
I20260620 08:09:33.706764 18338 leader_election.cc:288] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:33.706805 18343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:33.706923 18343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [term 1 LEADER]: Becoming Leader. State: Replica: 76cbfd0dfb924d26bfb24de278f5275f, State: Running, Role: LEADER
I20260620 08:09:33.707001 18343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } }
I20260620 08:09:33.707043 18338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:33.707314 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76cbfd0dfb924d26bfb24de278f5275f. Latest consensus state: current_term: 1 leader_uuid: "76cbfd0dfb924d26bfb24de278f5275f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } } }
I20260620 08:09:33.707389 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:33.707322 18345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76cbfd0dfb924d26bfb24de278f5275f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cbfd0dfb924d26bfb24de278f5275f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39683 } } }
I20260620 08:09:33.707557 18345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:33.707613 18349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:33.708137 18349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:33.709738 18349 catalog_manager.cc:1347] Generated new cluster ID: 867d3fa31dc94937a1b2eff57f001055
I20260620 08:09:33.709793 18349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:33.711303 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.705490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.12.132.254:39683)
0620 08:09:33.705704 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.705718 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.705768 (+ 50us) server_negotiation.cc:407] Connection header received
0620 08:09:33.705896 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.705902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.705987 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.706127 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:33.706297 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.707887 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.709183 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.709748 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.709865 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.710598 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:33.710602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:33.710602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:09:33.710836 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:33.710874 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:09:33.710888 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:33.710971 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:33.711029 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 08:09:33.711102 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260620 08:09:33.712415 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:09:33.719813 18349 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:33.720827 18349 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:33.725102 18349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76cbfd0dfb924d26bfb24de278f5275f: Generated new TSK 0
I20260620 08:09:33.725261 18349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:09:33.775908 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:33.737878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.12.132.254:39683)
0620 08:09:33.737938 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:33.737940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:33.738042 (+ 102us) server_negotiation.cc:407] Connection header received
0620 08:09:33.738083 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:33.738084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:33.738096 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:33.738174 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0620 08:09:33.738283 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.739041 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.739803 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.740103 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.740173 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.740332 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 08:09:33.775648 (+ 35316us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 08:09:33.775828 (+ 180us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37390: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 08:09:33.775983 18342 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37390: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 08:09:33.776199 18367 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:33.737794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39683 (local address 127.0.0.1:37390)
0620 08:09:33.737971 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:33.737981 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:33.738037 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:33.738177 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:33.738178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:33.738180 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 08:09:33.738224 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.738226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.739136 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.739137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.739720 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:33.739722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:33.740190 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:33.740191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:33.740234 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:33.740238 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 08:09:33.775806 (+ 35568us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 08:09:33.776096 (+ 290us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.132.254:39683: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260620 08:09:33.776585 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18271
2026-06-20T08:09:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (221 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:34 dist-test-slave-z0sp krb5kdc[18379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:34 dist-test-slave-z0sp krb5kdc[18379](info): set up 2 sockets
Jun 20 08:09:34 dist-test-slave-z0sp krb5kdc[18379](info): commencing operation
krb5kdc: starting...
W20260620 08:09:35.013149 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:35.136226 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44501
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:44501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:35.220817 18392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:35.221048 18392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:35.221086 18392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:35.222218 18392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:35.222283 18392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:35.222311 18392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:35.222342 18392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:35.222368 18392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:35.223814 18392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:44501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:35.224083 18392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:35.224303 18392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:35.229224 18400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.229326 18398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.229326 18397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:35.229728 18392 server_base.cc:1034] running on GCE node
I20260620 08:09:35.230013 18392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:35.230293 18392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:35.231472 18392 hybrid_clock.cc:648] HybridClock initialized: now 1781942975231467 us; error 38 us; skew 500 ppm
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.234252 18392 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:35.234676 18392 webserver.cc:460] Webserver started at http://127.12.132.254:38781/ using document root <none> and password file <none>
I20260620 08:09:35.234895 18392 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:35.234939 18392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:35.235072 18392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:35.236076 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "fa0cec7767b84e3e99d617d14f072f70"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "047ac8834cd9d8656db2ab302e97f845"
server_key_iv: "f26ebaee1d421a900a2520ec2ac8f6af"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.236423 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "fa0cec7767b84e3e99d617d14f072f70"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "047ac8834cd9d8656db2ab302e97f845"
server_key_iv: "f26ebaee1d421a900a2520ec2ac8f6af"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.237752 18392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:35.238675 18407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.238967 18392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:35.239143 18392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "fa0cec7767b84e3e99d617d14f072f70"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "047ac8834cd9d8656db2ab302e97f845"
server_key_iv: "f26ebaee1d421a900a2520ec2ac8f6af"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.239257 18392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:35.258185 18392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:35.260409 18392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:35.260632 18392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:35.265431 18392 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:44501
I20260620 08:09:35.265477 18459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:44501 every 8 connection(s)
I20260620 08:09:35.265820 18392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:35.266645 18460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:35.268901 18460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: Bootstrap starting.
I20260620 08:09:35.269423 18460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:35.269651 18460 log.cc:824] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:35.270346 18460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: No bootstrap required, opened a new log
I20260620 08:09:35.271185 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18392
I20260620 08:09:35.271275 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:35.271600 12819 external_mini_cluster.cc:1428] Setting key 2e50e2a966f3f24f4798811a04bdd26f
I20260620 08:09:35.272980 18460 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } }
I20260620 08:09:35.273156 18460 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:35.273238 18460 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa0cec7767b84e3e99d617d14f072f70, State: Initialized, Role: FOLLOWER
I20260620 08:09:35.273367 18460 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } }
I20260620 08:09:35.273469 18460 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:35.273520 18460 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:35.273571 18460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:35.274644 18460 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } }
I20260620 08:09:35.274753 18460 leader_election.cc:302] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa0cec7767b84e3e99d617d14f072f70; no voters:
I20260620 08:09:35.274888 18460 leader_election.cc:288] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:35.275084 18465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:35.275151 18460 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:35.275444 18465 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [term 1 LEADER]: Becoming Leader. State: Replica: fa0cec7767b84e3e99d617d14f072f70, State: Running, Role: LEADER
I20260620 08:09:35.275614 18465 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } }
I20260620 08:09:35.276475 18466 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa0cec7767b84e3e99d617d14f072f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } } }
I20260620 08:09:35.276634 18466 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:35.276612 18467 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa0cec7767b84e3e99d617d14f072f70. Latest consensus state: current_term: 1 leader_uuid: "fa0cec7767b84e3e99d617d14f072f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa0cec7767b84e3e99d617d14f072f70" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44501 } } }
I20260620 08:09:35.276684 18467 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70 [sys.catalog]: This master's current role is: LEADER
W20260620 08:09:35.277221 18479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:35.277278 18479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:35.277343 18480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:35.277498 18480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:35.279241 18480 catalog_manager.cc:1347] Generated new cluster ID: b29d322f6ec34ab789b05958e905ca95
I20260620 08:09:35.279299 18480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:35.284384 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.272423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.12.132.254:44501)
0620 08:09:35.272660 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.272669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.272710 (+ 41us) server_negotiation.cc:407] Connection header received
0620 08:09:35.272942 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.272949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.273016 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.273168 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.273985 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.275408 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.276606 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.277313 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.277715 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.280269 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.280273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.280274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.280285 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.282402 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.282797 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.282800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.282801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.282943 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.283147 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.283148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.283148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.283507 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.283699 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.283776 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.283911 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":135,"threads_started":1}
I20260620 08:09:35.290499 18480 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:35.291132 18480 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:35.300580 18480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa0cec7767b84e3e99d617d14f072f70: Generated new TSK 0
I20260620 08:09:35.300820 18480 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:35.333093 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:35.415324 18486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:35.415556 18486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:35.415594 18486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:35.416761 18486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:35.416826 18486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:35.416872 18486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:35.418390 18486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:35.418659 18486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:35.418883 18486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:35.424106 18494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.424194 18492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.424106 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:35.424577 18486 server_base.cc:1034] running on GCE node
I20260620 08:09:35.424772 18486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:35.425010 18486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:35.426198 18486 hybrid_clock.cc:648] HybridClock initialized: now 1781942975426183 us; error 23 us; skew 500 ppm
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.428737 18486 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:35.429229 18486 webserver.cc:460] Webserver started at http://127.12.132.193:36837/ using document root <none> and password file <none>
I20260620 08:09:35.429466 18486 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:35.429541 18486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:35.429677 18486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:35.430680 18486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "8ce3d5b091ac46df91c30f7143a43d57"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "d7ce6b9be2ad79c0ca82e63cdcef2af9"
server_key_iv: "f287a607ab8f5308bcb42bde47001868"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.431046 18486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "8ce3d5b091ac46df91c30f7143a43d57"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "d7ce6b9be2ad79c0ca82e63cdcef2af9"
server_key_iv: "f287a607ab8f5308bcb42bde47001868"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.432338 18486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:35.433168 18501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.433391 18486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:09:35.433490 18486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "8ce3d5b091ac46df91c30f7143a43d57"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "d7ce6b9be2ad79c0ca82e63cdcef2af9"
server_key_iv: "f287a607ab8f5308bcb42bde47001868"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.433588 18486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:35.446493 18486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:35.449007 18486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:35.449179 18486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:35.449399 18486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:35.449700 18486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:35.449755 18486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.449791 18486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:35.449817 18486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.455855 18486 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:44803
I20260620 08:09:35.455902 18614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:44803 every 8 connection(s)
I20260620 08:09:35.456213 18486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:35.458633 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18486
I20260620 08:09:35.458752 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:35.458951 12819 external_mini_cluster.cc:1428] Setting key fde441b1c88753eae0a8cc16f6c500d3
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:35.469519 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.457062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:58805 (local address 127.12.132.254:44501)
0620 08:09:35.457137 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.457140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.457377 (+ 237us) server_negotiation.cc:407] Connection header received
0620 08:09:35.458685 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.458687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.458704 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.458825 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.459931 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.460737 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.462478 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.462680 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.462746 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.465734 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.465736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.465737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.465746 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.467638 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.468303 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.468305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.468306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.468365 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.468786 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.468787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.468788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.468941 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.469039 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.469337 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.469422 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
I20260620 08:09:35.469673 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.457032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44501 (local address 127.12.132.193:58805)
0620 08:09:35.457250 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:35.457291 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:35.458503 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:35.458927 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:35.458933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:35.459440 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:35.459765 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.459792 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.460870 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.460871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.462358 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.462362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.462789 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.462790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.462963 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.463675 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:35.463686 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:35.465646 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:35.467769 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.467780 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.467783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.468196 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.468561 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.468562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.468563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.468701 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.469051 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:35.469057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:35.469187 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 08:09:35.469284 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 08:09:35.469426 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260620 08:09:35.470230 18615 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44501
I20260620 08:09:35.470323 18615 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:35.470463 18615 heartbeater.cc:511] Master 127.12.132.254:44501 requested a full tablet report, sending...
I20260620 08:09:35.471068 18424 ts_manager.cc:194] Registered new tserver with Master: 8ce3d5b091ac46df91c30f7143a43d57 (127.12.132.193:44803)
I20260620 08:09:35.471664 18424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:58805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:35.488194 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:35.570359 18620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:35.570574 18620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:35.570598 18620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:35.571825 18620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:35.571867 18620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:35.571894 18620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:35.573357 18620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:35.573575 18620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:35.573813 18620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:35.578685 18628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.578692 18626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.578692 18625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:35.578954 18620 server_base.cc:1034] running on GCE node
I20260620 08:09:35.579130 18620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:35.579311 18620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:35.580462 18620 hybrid_clock.cc:648] HybridClock initialized: now 1781942975580450 us; error 45 us; skew 500 ppm
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.583411 18620 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:35.583840 18620 webserver.cc:460] Webserver started at http://127.12.132.194:33931/ using document root <none> and password file <none>
I20260620 08:09:35.584069 18620 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:35.584138 18620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:35.584285 18620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:35.585397 18620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "ba70706d2dc84a1bbca0e061541adc8a"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "7b433b0ae62423c4c590e97322b396d7"
server_key_iv: "cfaf19c0376e79f83c2ce4dfd781e6b8"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.585755 18620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "ba70706d2dc84a1bbca0e061541adc8a"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "7b433b0ae62423c4c590e97322b396d7"
server_key_iv: "cfaf19c0376e79f83c2ce4dfd781e6b8"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.587102 18620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:35.588052 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.588371 18620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:35.588568 18620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "ba70706d2dc84a1bbca0e061541adc8a"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "7b433b0ae62423c4c590e97322b396d7"
server_key_iv: "cfaf19c0376e79f83c2ce4dfd781e6b8"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.588706 18620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:35.610796 18620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:35.613339 18620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:35.613511 18620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:35.613756 18620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:35.614084 18620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:35.614138 18620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.614179 18620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:35.614208 18620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.620678 18620 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36777
I20260620 08:09:35.620806 18748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36777 every 8 connection(s)
I20260620 08:09:35.621106 18620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:35.623356 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18620
I20260620 08:09:35.623474 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:35.623667 12819 external_mini_cluster.cc:1428] Setting key 51691120cc0e09eeefbac3590899bcfd
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:35.633143 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.621901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40671 (local address 127.12.132.254:44501)
0620 08:09:35.622003 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.622005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.622151 (+ 146us) server_negotiation.cc:407] Connection header received
0620 08:09:35.623131 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.623132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.623142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.623243 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.624025 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.624736 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.626322 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.626892 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.626981 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.629940 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.629942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.629942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.629951 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.631557 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.632025 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.632027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.632027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.632133 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.632450 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.632452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.632452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.632604 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.632684 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.632965 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.633048 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 08:09:35.633358 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.621824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44501 (local address 127.12.132.194:40671)
0620 08:09:35.622013 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:35.622053 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:35.623023 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:35.623315 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:35.623319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:35.623635 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:35.623890 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.623894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.624833 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.624834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.626174 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.626178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.627037 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.627038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.627220 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.627832 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:35.627846 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:35.629839 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:35.631674 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.631682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.631684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.631923 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.632234 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.632235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.632236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.632368 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.632674 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:35.632681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:35.632824 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 08:09:35.632944 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 08:09:35.633113 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260620 08:09:35.633926 18749 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44501
I20260620 08:09:35.634016 18749 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:35.634204 18749 heartbeater.cc:511] Master 127.12.132.254:44501 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
I20260620 08:09:35.634588 18424 ts_manager.cc:194] Registered new tserver with Master: ba70706d2dc84a1bbca0e061541adc8a (127.12.132.194:36777)
I20260620 08:09:35.634955 18424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:40671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:35.656320 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:35.737524 18754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:35.737713 18754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:35.737735 18754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:35.738818 18754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:35.738853 18754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:35.738879 18754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:35.740252 18754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:35.740448 18754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:35.740692 18754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:35.745805 18759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.745790 18762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:35.745826 18760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:35.746091 18754 server_base.cc:1034] running on GCE node
I20260620 08:09:35.746446 18754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:35.746654 18754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:35.747830 18754 hybrid_clock.cc:648] HybridClock initialized: now 1781942975747826 us; error 38 us; skew 500 ppm
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.750644 18754 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:35.751144 18754 webserver.cc:460] Webserver started at http://127.12.132.195:38729/ using document root <none> and password file <none>
I20260620 08:09:35.751384 18754 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:35.751428 18754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:35.751569 18754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:35.752635 18754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "d6a8433e4e5444cfa553f6597dafa66c"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "f52785ade0f6d60366517e226aa8feb3"
server_key_iv: "93bca60c205d5ecfff5453620a7b1d57"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.753037 18754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "d6a8433e4e5444cfa553f6597dafa66c"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "f52785ade0f6d60366517e226aa8feb3"
server_key_iv: "93bca60c205d5ecfff5453620a7b1d57"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.754331 18754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:35.755146 18769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.755393 18754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:35.755488 18754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "d6a8433e4e5444cfa553f6597dafa66c"
format_stamp: "Formatted at 2026-06-20 08:09:35 on dist-test-slave-z0sp"
server_key: "f52785ade0f6d60366517e226aa8feb3"
server_key_iv: "93bca60c205d5ecfff5453620a7b1d57"
server_key_version: "encryptionkey@0"
I20260620 08:09:35.755568 18754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:35.767366 18754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:35.770056 18754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:35.770166 18754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:35.770395 18754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:35.770715 18754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:35.770773 18754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.770824 18754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:35.770855 18754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:35.777177 18754 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:44369
I20260620 08:09:35.777215 18882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:44369 every 8 connection(s)
I20260620 08:09:35.777532 18754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:35.781788 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18754
I20260620 08:09:35.781932 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:35.782162 12819 external_mini_cluster.cc:1428] Setting key df0daf87cadcfc294c7b54084082d499
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:35.789139 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.778327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:58845 (local address 127.12.132.254:44501)
0620 08:09:35.778374 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.778377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.778634 (+ 257us) server_negotiation.cc:407] Connection header received
0620 08:09:35.779545 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.779547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.779557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.779609 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.780162 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.780828 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.782544 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.782782 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.782869 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.785591 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.785593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.785593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.785600 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.787247 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.787873 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.787899 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.787900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.787963 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.788339 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.788341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.788341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.788580 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.788656 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.788988 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.789069 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:35.789278 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.778319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44501 (local address 127.12.132.195:58845)
0620 08:09:35.778491 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:35.778525 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:35.779410 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:35.779664 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:35.779668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:35.779901 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:35.780081 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.780084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.780980 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.780982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.782381 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:35.782386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.782897 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.782898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:35.783077 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.783663 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:35.783673 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:35.785477 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:35.787372 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.787380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.787382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.787655 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.788075 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:35.788077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:35.788077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:35.788226 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:35.788679 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:35.788685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:35.788814 (+ 129us) client_negotiation.cc:769] Sending connection context
0620 08:09:35.788915 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 08:09:35.789073 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260620 08:09:35.789923 18883 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44501
I20260620 08:09:35.790011 18883 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:35.790144 18883 heartbeater.cc:511] Master 127.12.132.254:44501 requested a full tablet report, sending...
I20260620 08:09:35.790546 18424 ts_manager.cc:194] Registered new tserver with Master: d6a8433e4e5444cfa553f6597dafa66c (127.12.132.195:44369)
I20260620 08:09:35.790931 18424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:58845
I20260620 08:09:35.794463 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/test-admin.keytab.
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.814342 12819 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 08:09:35.816318 18890 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:35.815408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.12.132.193:44803)
0620 08:09:35.815595 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.815600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.815612 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:35.815638 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.815640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.815654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.815728 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.816224 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59690: BlockingRecv error: recv got EOF from 127.0.0.1:59690 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
W20260620 08:09:35.816471 12819 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc' of type FILE
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.817945 12819 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.819222 12819 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:35.828819 18890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.819888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.12.132.193:44803)
0620 08:09:35.819966 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.819973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.819984 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:35.820004 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.820005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.820014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.820080 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.820702 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.821425 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.822161 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.822484 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.822577 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.824865 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.824873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.824874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.824888 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.827241 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.827730 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.827734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.827735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.827802 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.828018 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.828019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.828019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.828277 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.828492 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.828619 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.828712 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260620 08:09:35.831231 18890 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:35.830501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.12.132.193:44803)
0620 08:09:35.830559 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.830560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.830657 (+ 97us) server_negotiation.cc:407] Connection header received
0620 08:09:35.830729 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.830730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.830742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.830853 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.831158 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59720: BlockingRecv error: recv got EOF from 127.0.0.1:59720 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260620 08:09:35.831344 12819 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc' of type FILE
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.832571 12819 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.833757 12819 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:35.842060 18890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.834506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.12.132.193:44803)
0620 08:09:35.834562 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.834564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.834572 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:35.834593 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.834594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.834607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.834675 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.835056 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.835618 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.836339 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.836618 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.836690 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.838899 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.838901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.838901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.838907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.840294 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.840949 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.840950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.840951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.841005 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.841314 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.841316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.841317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.841471 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.841608 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.841795 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.841955 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 08:09:35.844331 18890 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:35.843479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.12.132.193:44803)
0620 08:09:35.843579 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.843581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.843709 (+ 128us) server_negotiation.cc:407] Connection header received
0620 08:09:35.843795 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.843796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.843808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.843898 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.844250 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59750: BlockingRecv error: recv got EOF from 127.0.0.1:59750 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 08:09:35.844601 12819 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781942949664683-12819-0/krb5kdc/krb5cc' of type FILE
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.845894 12819 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:35.846949 12819 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:09:35.854872 18890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:35.847651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.12.132.193:44803)
0620 08:09:35.847708 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:35.847711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:35.847720 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:35.847802 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:35.847803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:35.847813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:35.847897 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:35.848279 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.848823 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.849513 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:35.849696 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:35.849801 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:35.852096 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:35.852097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:35.852098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:35.852105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:35.853718 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.854099 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.854101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.854101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.854157 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:35.854350 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:35.854352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:35.854352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:35.854454 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:35.854560 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:35.854689 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 08:09:35.854765 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:09:35.855551 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18486
I20260620 08:09:35.860669 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18620
I20260620 08:09:35.865032 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18754
I20260620 08:09:35.870832 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18392
2026-06-20T08:09:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1921 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18903](info): set up 2 sockets
Jun 20 08:09:35 dist-test-slave-z0sp krb5kdc[18903](info): commencing operation
krb5kdc: starting...
W20260620 08:09:36.937471 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab.
Entry for principal oryx/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab.
I20260620 08:09:37.070041 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37251
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab
--principal=oryx/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:37.152958 18916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:37.153208 18916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:37.153245 18916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:37.154350 18916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:37.154420 18916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:37.154444 18916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:37.154474 18916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:37.154505 18916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:37.155936 18916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab
--principal=oryx/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:37.156208 18916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:37.156430 18916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.161403 18921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.161417 18924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.161567 18922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.161895 18916 server_base.cc:1034] running on GCE node
I20260620 08:09:37.162175 18916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.162467 18916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.163666 18916 hybrid_clock.cc:648] HybridClock initialized: now 1781942977163638 us; error 47 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.170045 18916 init.cc:377] Logged in from keytab as oryx/127.12.132.254@KRBTEST.COM (short username oryx)
I20260620 08:09:37.170658 18916 webserver.cc:460] Webserver started at http://127.12.132.254:43353/ using document root <none> and password file <none>
I20260620 08:09:37.170943 18916 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.170993 18916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.171139 18916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:37.172433 18916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "da20f1d0b50541c596936809d58f6c31"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b67a265d877c675e2c5041b95408841"
server_key_iv: "9845abb9f820999494c9d52c35089839"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.172945 18916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "da20f1d0b50541c596936809d58f6c31"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b67a265d877c675e2c5041b95408841"
server_key_iv: "9845abb9f820999494c9d52c35089839"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.174578 18916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:37.175659 18931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.175935 18916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:37.176064 18916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "da20f1d0b50541c596936809d58f6c31"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b67a265d877c675e2c5041b95408841"
server_key_iv: "9845abb9f820999494c9d52c35089839"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.176160 18916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.188652 18916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:37.191656 18916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:37.191886 18916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:37.197036 18916 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:37251
I20260620 08:09:37.197072 18983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:37251 every 8 connection(s)
I20260620 08:09:37.197433 18916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:37.198405 18984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:37.200723 18984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Bootstrap starting.
I20260620 08:09:37.201300 18984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:37.201594 18984 log.cc:824] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:37.202503 18984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: No bootstrap required, opened a new log
I20260620 08:09:37.204077 18984 raft_consensus.cc:348] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } }
I20260620 08:09:37.204207 18984 raft_consensus.cc:374] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:37.204229 18984 raft_consensus.cc:729] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da20f1d0b50541c596936809d58f6c31, State: Initialized, Role: FOLLOWER
I20260620 08:09:37.204378 18984 consensus_queue.cc:260] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } }
I20260620 08:09:37.204469 18984 raft_consensus.cc:388] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:37.204550 18984 raft_consensus.cc:482] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:37.204599 18984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:37.205293 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 18916
I20260620 08:09:37.205399 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:37.205480 18984 raft_consensus.cc:504] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } }
I20260620 08:09:37.205607 18984 leader_election.cc:302] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da20f1d0b50541c596936809d58f6c31; no voters:
I20260620 08:09:37.205675 12819 external_mini_cluster.cc:1428] Setting key 114d884ff25dec5fc8ef2e31bf6aa26b
I20260620 08:09:37.205778 18984 leader_election.cc:288] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:37.205801 18987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:37.205910 18987 raft_consensus.cc:686] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 LEADER]: Becoming Leader. State: Replica: da20f1d0b50541c596936809d58f6c31, State: Running, Role: LEADER
I20260620 08:09:37.206001 18987 consensus_queue.cc:237] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } }
I20260620 08:09:37.206043 18984 sys_catalog.cc:564] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:37.206293 18987 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da20f1d0b50541c596936809d58f6c31. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } }
I20260620 08:09:37.206368 18987 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:37.206303 18988 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } }
I20260620 08:09:37.206418 18988 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
W20260620 08:09:37.207872 19005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:37.207949 19005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:37.208657 18993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:37.208838 18993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:37.210229 18993 catalog_manager.cc:1347] Generated new cluster ID: 0de3eb7b4ae340daa18eca021c05787c
I20260620 08:09:37.210278 18993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:37.219558 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.206321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.12.132.254:37251)
0620 08:09:37.206834 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.206843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.206873 (+ 30us) server_negotiation.cc:407] Connection header received
0620 08:09:37.206990 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.206996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.207036 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.207967 (+ 931us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.208071 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.209459 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.210511 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.211151 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.211576 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.215854 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.215858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.215858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.215870 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.217879 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.218291 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.218294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.218294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.218404 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.218624 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.218625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.218625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.218918 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.219151 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.219213 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.219338 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":75,"threads_started":1}
I20260620 08:09:37.226148 18993 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:37.226936 18993 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:37.231225 18993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Generated new TSK 0
I20260620 08:09:37.231412 18993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
I20260620 08:09:37.268784 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:37.348480 19010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:37.348696 19010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:37.348719 19010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:37.349869 19010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:37.349911 19010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:37.349937 19010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:37.351473 19010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:37.351729 19010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:37.352010 19010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.357266 19016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.357334 19015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.357332 19010 server_base.cc:1034] running on GCE node
W20260620 08:09:37.357288 19018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.357692 19010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.357959 19010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.359328 19010 hybrid_clock.cc:648] HybridClock initialized: now 1781942977359312 us; error 33 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.362229 19010 init.cc:377] Logged in from keytab as oryx/127.12.132.193@KRBTEST.COM (short username oryx)
I20260620 08:09:37.362751 19010 webserver.cc:460] Webserver started at http://127.12.132.193:35987/ using document root <none> and password file <none>
I20260620 08:09:37.362994 19010 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.363063 19010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.363224 19010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:37.364248 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "28e3bb56273b43a9aa0437c42b9f7e23"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "27e8c69f8d2b4264eb9b1bcf1849d13b"
server_key_iv: "6fceaa1703d6787cfe7e496b498a5803"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.364658 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "28e3bb56273b43a9aa0437c42b9f7e23"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "27e8c69f8d2b4264eb9b1bcf1849d13b"
server_key_iv: "6fceaa1703d6787cfe7e496b498a5803"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.366114 19010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:37.366998 19025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.367336 19010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:37.367507 19010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "28e3bb56273b43a9aa0437c42b9f7e23"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "27e8c69f8d2b4264eb9b1bcf1849d13b"
server_key_iv: "6fceaa1703d6787cfe7e496b498a5803"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.367630 19010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.377470 19010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:37.379663 19010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:37.379822 19010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:37.380048 19010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:37.380352 19010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:37.380409 19010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.380443 19010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:37.380470 19010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.386749 19010 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:40273
I20260620 08:09:37.386796 19138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:40273 every 8 connection(s)
I20260620 08:09:37.387089 19010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:37.393826 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19010
I20260620 08:09:37.393944 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:37.394522 12819 external_mini_cluster.cc:1428] Setting key 0dc2ecb5a701684ec1b131e53263fb11
I20260620 08:09:37.396895 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.387802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:53263 (local address 127.12.132.254:37251)
0620 08:09:37.387885 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.387888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.388038 (+ 150us) server_negotiation.cc:407] Connection header received
0620 08:09:37.388600 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.388601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.388616 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.388667 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.389607 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.390113 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.391110 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.391329 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.391418 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.393734 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.393736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.393737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.393744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.395435 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.395861 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.395862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.395863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.395923 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.396183 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.396184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.396184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.396300 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.396377 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.396714 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.396804 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 08:09:37.397078 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.387708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:53263)
0620 08:09:37.387878 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.387951 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.388456 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.388731 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.388734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.389304 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.389485 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.389489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.390210 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.390211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.391012 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.391014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.391430 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.391431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.391564 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.392073 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.392081 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.393630 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.395523 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.395531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.395533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.395769 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.395993 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.395993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.395994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.396106 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.396367 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.396374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.396567 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.396690 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.396845 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260620 08:09:37.397589 19139 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:37.397699 19139 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:37.397890 19139 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:37.398391 18948 ts_manager.cc:194] Registered new tserver with Master: 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273)
I20260620 08:09:37.398996 18948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:53263
WARNING: no policy specified for oryx/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
I20260620 08:09:37.423759 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:37.507002 19144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:37.507222 19144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:37.507266 19144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:37.508420 19144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:37.508471 19144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:37.508553 19144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:37.509950 19144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:37.510126 19144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:37.510293 19144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.515242 19149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.515277 19152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.515277 19150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.515605 19144 server_base.cc:1034] running on GCE node
I20260620 08:09:37.515799 19144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.516036 19144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.517197 19144 hybrid_clock.cc:648] HybridClock initialized: now 1781942977517184 us; error 30 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.519565 19144 init.cc:377] Logged in from keytab as oryx/127.12.132.194@KRBTEST.COM (short username oryx)
I20260620 08:09:37.520032 19144 webserver.cc:460] Webserver started at http://127.12.132.194:44371/ using document root <none> and password file <none>
I20260620 08:09:37.520251 19144 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.520318 19144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.520469 19144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:37.521481 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "fa99f38679a04959aa331e13f05064c8"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "ee60ae7fbc2048d9a63277e8904ee0ea"
server_key_iv: "ac300948316cef99baab87fecc646f17"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.521826 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "fa99f38679a04959aa331e13f05064c8"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "ee60ae7fbc2048d9a63277e8904ee0ea"
server_key_iv: "ac300948316cef99baab87fecc646f17"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.523329 19144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:37.524464 19159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.524772 19144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:37.524904 19144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "fa99f38679a04959aa331e13f05064c8"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "ee60ae7fbc2048d9a63277e8904ee0ea"
server_key_iv: "ac300948316cef99baab87fecc646f17"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.525010 19144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.558399 19144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:37.561147 19144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:37.561306 19144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:37.561558 19144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:37.561879 19144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:37.561936 19144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.561972 19144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:37.562000 19144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.567734 19144 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:44381
I20260620 08:09:37.567773 19272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:44381 every 8 connection(s)
I20260620 08:09:37.568107 19144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:37.569866 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19144
I20260620 08:09:37.570007 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:37.570196 12819 external_mini_cluster.cc:1428] Setting key c44a8455960a62f38c185dc2ba64cac0
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.12.132.195@KRBTEST.COM; defaulting to no policy
I20260620 08:09:37.578493 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.568889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:38449 (local address 127.12.132.254:37251)
0620 08:09:37.568956 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.568958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.569158 (+ 200us) server_negotiation.cc:407] Connection header received
0620 08:09:37.569705 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.569706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.569719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.569794 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.571196 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.571695 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.572837 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.573055 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.573141 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.575517 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.575519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.575519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.575527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.577036 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.577436 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.577438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.577438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.577502 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.577758 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.577758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.577759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.577871 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.577946 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.578280 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.578392 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:37.578585 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.568873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:38449)
0620 08:09:37.569028 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.569070 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.569589 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.569859 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.569863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.570757 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.571074 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.571079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.571796 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.571797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.572750 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.572752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.573149 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.573150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.573283 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.573808 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.573816 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.575403 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.577135 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.577141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.577143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.577359 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.577571 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.577572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.577572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.577682 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.577937 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.577943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.578081 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.578198 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.578335 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260620 08:09:37.579046 19273 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:37.579121 19273 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:37.579309 19273 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:37.579756 18948 ts_manager.cc:194] Registered new tserver with Master: fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
I20260620 08:09:37.580232 18948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:38449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
I20260620 08:09:37.599812 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:37.680537 19278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:37.680742 19278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:37.680765 19278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:37.681869 19278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:37.681907 19278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:37.681941 19278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:37.683344 19278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:37.683564 19278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:37.683809 19278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.688731 19286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.688737 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.688763 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.688822 19278 server_base.cc:1034] running on GCE node
I20260620 08:09:37.689096 19278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.689278 19278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.690419 19278 hybrid_clock.cc:648] HybridClock initialized: now 1781942977690406 us; error 29 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.692847 19278 init.cc:377] Logged in from keytab as oryx/127.12.132.195@KRBTEST.COM (short username oryx)
I20260620 08:09:37.693286 19278 webserver.cc:460] Webserver started at http://127.12.132.195:40321/ using document root <none> and password file <none>
I20260620 08:09:37.693512 19278 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.693580 19278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.693730 19278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:37.694757 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "70b5607f209c4a03b80b63dd3c663682"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "e8439f1c934260b4e7e4878d1173bbb5"
server_key_iv: "26d95da4faa316cf142b3fbd18e6b749"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.695132 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "70b5607f209c4a03b80b63dd3c663682"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "e8439f1c934260b4e7e4878d1173bbb5"
server_key_iv: "26d95da4faa316cf142b3fbd18e6b749"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.696573 19278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:37.697331 19293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.697522 19278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:37.697618 19278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "70b5607f209c4a03b80b63dd3c663682"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "e8439f1c934260b4e7e4878d1173bbb5"
server_key_iv: "26d95da4faa316cf142b3fbd18e6b749"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.697706 19278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.705607 19278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:37.707974 19278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:37.708168 19278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:37.708410 19278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:37.708738 19278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:37.708793 19278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.708834 19278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:37.708860 19278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.714911 19278 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:33631
I20260620 08:09:37.714952 19406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:33631 every 8 connection(s)
I20260620 08:09:37.715263 19278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:37.724332 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.715912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:43693 (local address 127.12.132.254:37251)
0620 08:09:37.715956 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.715958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.716144 (+ 186us) server_negotiation.cc:407] Connection header received
0620 08:09:37.716651 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.716652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.716661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.716724 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.717578 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.718061 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.719029 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.719207 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.719262 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.721284 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.721285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.721286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.721292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.722713 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.723302 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.723303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.723304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.723357 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.723702 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.723703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.723704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.723827 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.723905 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.724169 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.724243 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:37.724558 19409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.715865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:43693)
0620 08:09:37.716012 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.716051 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.716562 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.716774 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.716777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.717311 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.717490 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.717494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.718185 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.718186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.718950 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.718952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.719279 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.719280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.719373 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.719877 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.719884 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.721201 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.722845 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.722859 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.722861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.723202 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.723463 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.723465 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.723466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.723605 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.723898 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.723903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.724029 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.724133 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.724287 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260620 08:09:37.724977 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19278
I20260620 08:09:37.725106 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:37.725319 12819 external_mini_cluster.cc:1428] Setting key c269b536b9684a9ecdceada73b59919f
I20260620 08:09:37.725405 19407 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:37.725589 19407 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:37.725873 19407 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:37.726446 18948 ts_manager.cc:194] Registered new tserver with Master: 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:37.726925 18948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:43693
I20260620 08:09:37.737627 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.132.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.253.keytab.
Entry for principal oryx/127.12.132.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.253.keytab.
I20260620 08:09:37.763648 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:36483
--webserver_interface=127.12.132.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.253.keytab
--principal=oryx/127.12.132.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37251,127.12.132.253:36483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:37.845876 19412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:37.846158 19412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:37.846194 19412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:37.848089 19412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:37.848133 19412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:37.848157 19412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:37.848181 19412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:37.848202 19412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:37.850713 19412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37251,127.12.132.253:36483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.253.keytab
--principal=oryx/127.12.132.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:36483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.132.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
W20260620 08:09:37.850858 19412 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.132.254:37251,127.12.132.253:36483'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 08:09:37.851028 19412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:37.851259 19412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.856709 19420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.856709 19417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.856853 19418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.857980 19412 server_base.cc:1034] running on GCE node
I20260620 08:09:37.858253 19412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.858587 19412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.859760 19412 hybrid_clock.cc:648] HybridClock initialized: now 1781942977859736 us; error 42 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.862340 19412 init.cc:377] Logged in from keytab as oryx/127.12.132.253@KRBTEST.COM (short username oryx)
I20260620 08:09:37.862792 19412 webserver.cc:460] Webserver started at http://127.12.132.253:40731/ using document root <none> and password file <none>
I20260620 08:09:37.863034 19412 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.863078 19412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.863212 19412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:37.864224 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/instance:
uuid: "c11b170239114a1ebed558dd73006ee4"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b44595ce977efd729f3462eb7bcb04a"
server_key_iv: "b3fce1c41f2895c04e9f892099f83df5"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.864611 19412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal/instance:
uuid: "c11b170239114a1ebed558dd73006ee4"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b44595ce977efd729f3462eb7bcb04a"
server_key_iv: "b3fce1c41f2895c04e9f892099f83df5"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.865990 19412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:37.866724 19427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.866972 19412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:37.867063 19412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "c11b170239114a1ebed558dd73006ee4"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b44595ce977efd729f3462eb7bcb04a"
server_key_iv: "b3fce1c41f2895c04e9f892099f83df5"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.867153 19412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.878080 19412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:37.881057 19412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:37.881294 19412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:37.887059 19412 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.253:36483
I20260620 08:09:37.887104 19479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.253:36483 every 8 connection(s)
I20260620 08:09:37.887521 19412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
I20260620 08:09:37.888828 19480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:37.891129 19480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:09:37.892606 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19412
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.253@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
I20260620 08:09:37.902825 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.892023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.12.132.254:37251)
0620 08:09:37.892165 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.892168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.892300 (+ 132us) server_negotiation.cc:407] Connection header received
0620 08:09:37.892872 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.892874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.892894 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.892947 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.893804 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.894510 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.895413 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.895659 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.895726 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.898786 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.898789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.898790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.898805 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.900712 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.901581 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.901582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.901583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.901642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.901928 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.901929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.901929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.902131 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.902210 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.902566 (+ 356us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.902688 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260620 08:09:37.903509 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.891946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55436)
0620 08:09:37.892162 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.892205 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.892748 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.893036 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.893041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.893428 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.893709 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.893714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.894606 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.894607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.895302 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.895304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.895741 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.895743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.895900 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.896600 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.896609 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.898635 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.900825 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.900838 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.900842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.901281 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.901724 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.901725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.901726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.901850 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.902210 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.902218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.902377 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.902986 (+ 609us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.903175 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260620 08:09:37.904278 19480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:09:37.904456 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.893097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.12.132.253:36483)
0620 08:09:37.893265 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.893269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.893305 (+ 36us) server_negotiation.cc:407] Connection header received
0620 08:09:37.893374 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.893376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.893394 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.893488 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.894165 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.895491 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.896239 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.896675 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.896772 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.900232 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.900239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.900239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.900246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.902361 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.902793 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.902796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.902796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.902864 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.903377 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.903378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.903378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.903784 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.903939 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.904045 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.904133 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.253@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
I20260620 08:09:37.911862 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.904638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.12.132.253:36483)
0620 08:09:37.904671 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.904672 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.904884 (+ 212us) server_negotiation.cc:407] Connection header received
0620 08:09:37.904952 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.904953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.904965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.905020 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.905365 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.905875 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.906715 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.907026 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.907103 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.909343 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.909344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.909345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.909352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.910707 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.911080 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.911081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.911081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.911134 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.911321 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.911322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.911322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.911431 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.911502 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.911648 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.911741 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260620 08:09:37.911919 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.904606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58564)
0620 08:09:37.904811 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.904821 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.904897 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.905024 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.905024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.905208 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.905284 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.905287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.905969 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.905970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.906626 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.906628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.907098 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.907099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.907174 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.907616 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.907623 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.909222 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.910827 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.910828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.910828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.910996 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.911197 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.911198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.911198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.911275 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.911494 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.911495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.911535 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.911627 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.911790 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":67}
I20260620 08:09:37.913704 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Bootstrap starting.
I20260620 08:09:37.914737 19480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:37.915068 19480 log.cc:824] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:37.915868 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: No bootstrap required, opened a new log
I20260620 08:09:37.918232 19480 raft_consensus.cc:348] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } }
I20260620 08:09:37.918401 19480 raft_consensus.cc:374] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:37.918437 19480 raft_consensus.cc:729] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11b170239114a1ebed558dd73006ee4, State: Initialized, Role: FOLLOWER
I20260620 08:09:37.918556 19480 consensus_queue.cc:260] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } }
I20260620 08:09:37.918799 19486 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } } }
I20260620 08:09:37.918861 19480 sys_catalog.cc:564] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:37.918875 19486 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:09:37.920811 19497 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:37.920863 19497 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:37.922168 19412 master_runner.cc:428] Detected that this master c11b170239114a1ebed558dd73006ee4 is joining an existing cluster
I20260620 08:09:37.922233 19412 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.132.253:36483
I20260620 08:09:37.930109 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.923587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58572 (local address 127.12.132.253:36483)
0620 08:09:37.923621 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.923622 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.923629 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:37.923645 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.923645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.923652 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.923698 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.924058 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.924788 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.925804 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.926074 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.926110 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.927271 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.927272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.927272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.927278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.928898 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929345 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.929345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.929346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.929393 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929576 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.929577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.929577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.929695 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.929736 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.929911 (+ 175us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.929999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:37.930284 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.923209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.12.132.254:37251)
0620 08:09:37.923253 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.923255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.923426 (+ 171us) server_negotiation.cc:407] Connection header received
0620 08:09:37.923468 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.923469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.923481 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.923538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.924197 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.924942 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.925890 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.926176 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.926214 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.927308 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.927309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.927310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.927316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.929060 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929505 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.929506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.929507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.929558 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929760 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.929761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.929762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.929890 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.929931 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.930112 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.930190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:37.930461 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.923411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58572)
0620 08:09:37.923537 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.923546 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.923587 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.923700 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.923701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.923925 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.923990 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.923992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.924863 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.924864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.925720 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.925722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.926191 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.926192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.926246 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.926693 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.926698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.927181 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.929016 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.929017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.929270 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.929461 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.929462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.929512 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.929776 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.929777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.929811 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.930204 (+ 393us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.930281 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260620 08:09:37.931137 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.923172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55444)
0620 08:09:37.923321 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.923333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.923400 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.923551 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.923552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.923740 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.924109 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.924112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.925019 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.925020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.925815 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.925816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.926283 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.926284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.926343 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.926868 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.926872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.927220 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.929135 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.929136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.929378 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.929629 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.929630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.929630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.929687 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.929976 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.929977 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.930009 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.930983 (+ 974us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.931070 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.254@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
I20260620 08:09:37.939823 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.933225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58578 (local address 127.12.132.253:36483)
0620 08:09:37.933281 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.933282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.933441 (+ 159us) server_negotiation.cc:407] Connection header received
0620 08:09:37.933527 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.933528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.933538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.933586 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.933974 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.934395 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.935138 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.935331 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.935401 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.937319 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.937321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.937321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.937328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.938660 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.939056 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.939057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.939058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.939117 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.939314 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.939315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.939315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.939420 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.939498 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.939669 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.939739 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:37.939869 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.933172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58578)
0620 08:09:37.933331 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.933364 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.933464 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.933589 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.933592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.933841 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.933884 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.933885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.934474 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.934475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.935045 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.935047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.935401 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.935402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.935504 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.935924 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.935930 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.937233 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.938759 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.938766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.938768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.938981 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.939183 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.939184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.939184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.939243 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.939492 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.939499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.939547 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.939645 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.939756 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260620 08:09:37.940349 18948 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.132.253:36483: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c11b170239114a1ebed558dd73006ee4"
member_type: NON_VOTER
last_known_addr {
host: "127.12.132.253"
port: 36483
}
attrs {
promote: true
}
}
dest_uuid: "da20f1d0b50541c596936809d58f6c31"
cas_config_opid_index: -1
I20260620 08:09:37.940704 18948 consensus_queue.cc:237] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } }
I20260620 08:09:37.941123 19507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c11b170239114a1ebed558dd73006ee4 (127.12.132.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } }
I20260620 08:09:37.941403 19454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:37.942163 19507 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.132.253:36483
I20260620 08:09:37.942245 19506 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } }
I20260620 08:09:37.942319 19454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 08:09:37.942325 19506 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:37.942705 19412 master.cc:537] Master@127.12.132.253:36483 shutting down...
I20260620 08:09:37.942696 19506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 08:09:37.944137 18932 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260620 08:09:37.946043 19412 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:37.946117 19412 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:37.946153 19412 tablet_replica.cc:330] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: stopping tablet replica
I20260620 08:09:37.952749 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.952617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58586)
0620 08:09:37.952661 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.952717 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:36483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260620 08:09:37.952903 18932 consensus_peers.cc:487] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 -> Peer c11b170239114a1ebed558dd73006ee4 (127.12.132.253:36483): Couldn't send request to peer c11b170239114a1ebed558dd73006ee4. Status: Network error: Client connection negotiation failed: client connection to 127.12.132.253:36483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 08:09:37.959270 19412 master.cc:552] Master@127.12.132.253:36483 shutdown complete.
I20260620 08:09:37.959321 19412 master_runner.cc:305] Clearing existing system tablet
I20260620 08:09:37.959450 19412 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.959499 19412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.960230 19412 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260620 08:09:37.960738 19511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.960929 19412 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:37.960990 19412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.961014 19412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "c11b170239114a1ebed558dd73006ee4"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b44595ce977efd729f3462eb7bcb04a"
server_key_iv: "b3fce1c41f2895c04e9f892099f83df5"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.961752 19412 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:37.963286 19412 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 08:09:37.963354 19412 log.cc:1196] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 08:09:37.963549 19412 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Deleting consensus metadata
I20260620 08:09:37.963696 19412 master_runner.cc:315] Copying system tablet from 127.12.132.254:37251
I20260620 08:09:37.964293 19412 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: tablet copy: Beginning tablet copy session from remote peer at address 127.12.132.254:37251
I20260620 08:09:37.970847 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.964717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.12.132.254:37251)
0620 08:09:37.964769 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:37.964771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:37.964782 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:37.964871 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:37.964872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:37.964882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:37.964937 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:37.965212 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.965981 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.966689 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.966996 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.967064 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.967973 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:37.967974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:37.967975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:37.967981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:37.969443 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.969853 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.969854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.969854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.969924 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:37.970136 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:37.970137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:37.970137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:37.970265 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:37.970341 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:37.970650 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 08:09:37.970729 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:09:37.971036 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:37.964587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55454)
0620 08:09:37.964717 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:37.964729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:37.964789 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:37.964942 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:37.964942 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:37.965089 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:37.965142 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.965144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.966063 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.966064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.966602 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:37.966603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:37.967068 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:37.967069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:37.967116 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:37.967591 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:37.967599 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:37.967894 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:37.969545 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.969545 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.969546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.969703 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.970011 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:37.970012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:37.970012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:37.970070 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:37.970420 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:37.970422 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:37.970496 (+ 74us) client_negotiation.cc:769] Sending connection context
0620 08:09:37.970604 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 08:09:37.970904 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260620 08:09:37.971400 18968 tablet_copy_service.cc:140] P da20f1d0b50541c596936809d58f6c31: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c11b170239114a1ebed558dd73006ee4 ({username='oryx', principal='oryx/127.12.132.253@KRBTEST.COM'} at 127.0.0.1:55454)
I20260620 08:09:37.971480 18968 tablet_copy_service.cc:161] P da20f1d0b50541c596936809d58f6c31: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c11b170239114a1ebed558dd73006ee4 at {username='oryx', principal='oryx/127.12.132.253@KRBTEST.COM'} at 127.0.0.1:55454: session id = c11b170239114a1ebed558dd73006ee4-00000000000000000000000000000000
I20260620 08:09:37.972172 18968 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 08:09:37.972783 19412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 08:09:37.973933 19412 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: tablet copy: Starting download of 0 data blocks...
I20260620 08:09:37.974082 19412 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: tablet copy: Starting download of 1 WAL segments...
I20260620 08:09:37.974917 19412 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 08:09:37.975845 18968 tablet_copy_service.cc:342] P da20f1d0b50541c596936809d58f6c31: Request end of tablet copy session c11b170239114a1ebed558dd73006ee4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.132.253@KRBTEST.COM'} at 127.0.0.1:55454
I20260620 08:09:37.975900 18968 tablet_copy_service.cc:433] P da20f1d0b50541c596936809d58f6c31: ending tablet copy session c11b170239114a1ebed558dd73006ee4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c11b170239114a1ebed558dd73006ee4
I20260620 08:09:37.977998 19412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:37.979071 19518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:37.979121 19519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.981748 19412 server_base.cc:1034] running on GCE node
W20260620 08:09:37.981781 19521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:37.981920 19412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:37.982082 19412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:37.983203 19412 hybrid_clock.cc:648] HybridClock initialized: now 1781942977983194 us; error 25 us; skew 500 ppm
Jun 20 08:09:37 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:37.984423 19412 init.cc:377] Logged in from keytab as oryx/127.12.132.253@KRBTEST.COM (short username oryx)
I20260620 08:09:37.984726 19412 webserver.cc:460] Webserver started at http://127.12.132.253:40197/ using document root <none> and password file <none>
I20260620 08:09:37.984861 19412 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:37.984899 19412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:37.985478 19412 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:37.986002 19528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:37.986140 19412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:37.986184 19412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "c11b170239114a1ebed558dd73006ee4"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "3b44595ce977efd729f3462eb7bcb04a"
server_key_iv: "b3fce1c41f2895c04e9f892099f83df5"
server_key_version: "encryptionkey@0"
I20260620 08:09:37.986236 19412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.023141 19412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.024986 19412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.028081 19412 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.253:36483
I20260620 08:09:38.028228 19580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.253:36483 every 8 connection(s)
I20260620 08:09:38.028426 19412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
I20260620 08:09:38.029337 19581 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:09:38.029907 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Bootstrap starting.
I20260620 08:09:38.033015 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:09:38.033262 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: Bootstrap complete.
I20260620 08:09:38.033447 19581 raft_consensus.cc:348] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } }
I20260620 08:09:38.033535 19581 raft_consensus.cc:729] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c11b170239114a1ebed558dd73006ee4, State: Initialized, Role: LEARNER
I20260620 08:09:38.033574 19581 consensus_queue.cc:260] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } }
I20260620 08:09:38.033751 19581 sys_catalog.cc:564] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:38.033746 19584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } }
I20260620 08:09:38.033869 19584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: LEARNER
I20260620 08:09:38.035295 19595 catalog_manager.cc:1259] Loaded cluster ID: 0de3eb7b4ae340daa18eca021c05787c
I20260620 08:09:38.035347 19595 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: loading cluster ID for follower catalog manager: success
I20260620 08:09:38.036219 19595 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: acquiring CA information for follower catalog manager: success
I20260620 08:09:38.036535 19595 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:09:38.064478 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.057118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.12.132.253:36483)
0620 08:09:38.057282 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.057284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.057296 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:38.057323 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.057324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.057336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.057426 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.058038 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.059103 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.059815 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.060157 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.060238 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.061503 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.061504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.061505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.061516 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.063430 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.063760 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.063761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.063762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.063818 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.063998 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.063999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.063999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.064119 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.064213 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.064286 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.064357 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.12.132.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.252.keytab.
Entry for principal oryx/127.12.132.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.252.keytab.
I20260620 08:09:38.092725 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:35309
--webserver_interface=127.12.132.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.252.keytab
--principal=oryx/127.12.132.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:38.179517 19599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:38.179734 19599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:38.179771 19599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:38.181171 19599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:38.181236 19599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:38.181264 19599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:38.181295 19599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:38.181321 19599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:38.182806 19599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.252.keytab
--principal=oryx/127.12.132.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:35309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.132.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:38.183072 19599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:38.183290 19599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:38.188118 19607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.188357 19604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.188467 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.188479 19599 server_base.cc:1034] running on GCE node
I20260620 08:09:38.188838 19599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:38.189136 19599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:38.190305 19599 hybrid_clock.cc:648] HybridClock initialized: now 1781942978190307 us; error 42 us; skew 500 ppm
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:38.192826 19599 init.cc:377] Logged in from keytab as oryx/127.12.132.252@KRBTEST.COM (short username oryx)
I20260620 08:09:38.193307 19599 webserver.cc:460] Webserver started at http://127.12.132.252:45955/ using document root <none> and password file <none>
I20260620 08:09:38.193547 19599 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.193596 19599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.193737 19599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:38.194706 19599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/instance:
uuid: "9b66ce99293c4d31949a86e20a77c454"
format_stamp: "Formatted at 2026-06-20 08:09:38 on dist-test-slave-z0sp"
server_key: "2f3b22b5284f4b703e70c7eae619c0e7"
server_key_iv: "e0927d7b0f42adb5ee770ba871c6a727"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.194996 19599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal/instance:
uuid: "9b66ce99293c4d31949a86e20a77c454"
format_stamp: "Formatted at 2026-06-20 08:09:38 on dist-test-slave-z0sp"
server_key: "2f3b22b5284f4b703e70c7eae619c0e7"
server_key_iv: "e0927d7b0f42adb5ee770ba871c6a727"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.196234 19599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:38.197949 19614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.198156 19599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.198297 19599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "9b66ce99293c4d31949a86e20a77c454"
format_stamp: "Formatted at 2026-06-20 08:09:38 on dist-test-slave-z0sp"
server_key: "2f3b22b5284f4b703e70c7eae619c0e7"
server_key_iv: "e0927d7b0f42adb5ee770ba871c6a727"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.198386 19599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.222705 19599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.225080 19599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:38.225239 19599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.230222 19599 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.252:35309
I20260620 08:09:38.230293 19666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.252:35309 every 8 connection(s)
I20260620 08:09:38.230898 19599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
I20260620 08:09:38.231950 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:38.233556 19667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:09:38.239053 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19599
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.252@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:38.243455 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.234452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.12.132.254:37251)
0620 08:09:38.234535 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.234537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.234625 (+ 88us) server_negotiation.cc:407] Connection header received
0620 08:09:38.235185 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.235186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.235196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.235252 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.235893 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.236380 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.237419 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.237626 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.237694 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.240547 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.240548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.240549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.240557 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.242054 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.242455 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.242456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.242457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.242514 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.242755 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.242755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.242756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.242852 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.242933 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.243227 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.243348 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:09:38.243544 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.234364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55470)
0620 08:09:38.234523 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.234557 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.235075 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.235364 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.235367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.235611 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.235807 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.235811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.236479 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.236480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.237319 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.237321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.237789 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.237791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.237972 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.238735 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.238744 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.240431 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.242139 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.242144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.242146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.242356 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.242573 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.242574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.242574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.242692 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.242948 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.242954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.243068 (+ 114us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.243191 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.243350 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260620 08:09:38.244086 19667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.252@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
I20260620 08:09:38.248229 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.239586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.12.132.252:35309)
0620 08:09:38.239756 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.239761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.239782 (+ 21us) server_negotiation.cc:407] Connection header received
0620 08:09:38.239810 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.239812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.239828 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.239909 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.240557 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.241493 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.242422 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.242840 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.242889 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.245469 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.245476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.245476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.245482 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.247047 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.247399 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.247401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.247401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.247461 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.247662 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.247663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.247663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.247814 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.247966 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.248041 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.248123 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260620 08:09:38.251335 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.244326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58632)
0620 08:09:38.244375 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.244386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.244425 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.244576 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.244577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.244703 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.244741 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.244742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.245512 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.245513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.246151 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.246153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.246708 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.246709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.246762 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.247218 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.247222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.248552 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.250185 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.250187 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.250188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.250464 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.250763 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.250764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.250764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.250847 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.251119 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.251119 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.251153 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.251204 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.251280 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 08:09:38.251354 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.244413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.12.132.253:36483)
0620 08:09:38.244460 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.244462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.244469 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:38.244512 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.244513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.244526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.244576 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.244817 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.245394 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.246257 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.246591 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.246669 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.248620 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.248621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.248621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.248626 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.250003 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.250591 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.250592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.250592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.250654 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.250940 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.250941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.250941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.251052 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.251114 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.251223 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.251287 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:09:38.251929 19667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.252@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
I20260620 08:09:38.258904 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.252299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56226)
0620 08:09:38.252427 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.252442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.252550 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.252765 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.252803 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.252969 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.253005 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.253007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.253791 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.253792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.254310 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.254311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.254698 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.254698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.254745 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.255134 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.255138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.256270 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.257873 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.257874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.257874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.258032 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.258241 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.258242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.258242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.258292 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.258549 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.258550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.258587 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.258654 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.258753 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260620 08:09:38.258953 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.252572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.12.132.252:35309)
0620 08:09:38.252642 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.252644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.252651 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:38.252671 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.252671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.252683 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.252763 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.253158 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.253685 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.254393 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.254615 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.254698 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.256369 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.256370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.256371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.256378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.257777 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.258115 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.258116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.258117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.258171 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.258372 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.258373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.258373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.258484 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.258557 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.258678 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.258800 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":46}
I20260620 08:09:38.260251 19667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Bootstrap starting.
I20260620 08:09:38.261067 19667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:38.261304 19667 log.cc:824] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:38.262111 19667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: No bootstrap required, opened a new log
I20260620 08:09:38.263614 19667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } }
I20260620 08:09:38.263733 19667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:38.263756 19667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b66ce99293c4d31949a86e20a77c454, State: Initialized, Role: FOLLOWER
I20260620 08:09:38.263881 19667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } }
I20260620 08:09:38.264142 19672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } } }
I20260620 08:09:38.264199 19667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:38.264247 19672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:09:38.265805 19683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:09:38.265864 19683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:09:38.267990 19599 master_runner.cc:428] Detected that this master 9b66ce99293c4d31949a86e20a77c454 is joining an existing cluster
I20260620 08:09:38.268025 19599 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.132.252:35309
I20260620 08:09:38.276432 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.268963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.12.132.253:36483)
0620 08:09:38.269037 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.269039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.269153 (+ 114us) server_negotiation.cc:407] Connection header received
0620 08:09:38.269255 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.269256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.269265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.269297 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.270004 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.270697 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.271815 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.272028 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.272059 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.273608 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.273610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.273610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.273616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.275177 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275655 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.275656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.275657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.275713 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275935 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.275936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.275936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.276058 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.276117 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.276284 (+ 167us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.276346 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:09:38.276463 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.268942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58642)
0620 08:09:38.269087 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.269097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.269178 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.269488 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.269489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.269734 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.269837 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.269838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.270801 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.270802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.271718 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.271720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.272123 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.272124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.272204 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.272891 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.272897 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.273505 (+ 608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.275276 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.275277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.275587 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.275787 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.275788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.275862 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.276140 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.276141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.276179 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.276248 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.276342 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":52,"thread_start_us":74,"threads_started":1}
I20260620 08:09:38.276964 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.269140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56240)
0620 08:09:38.269231 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.269239 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.269279 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.269468 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.269469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.269726 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.269837 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.269838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.270805 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.270806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.271718 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.271720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.272174 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.272174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.272230 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.272891 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.272897 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.273505 (+ 608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.275628 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.275628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.275819 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.275985 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.275986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.276036 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.276310 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.276311 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.276343 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.276410 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.276859 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":40,"thread_start_us":54,"threads_started":1}
I20260620 08:09:38.277139 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.269226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.12.132.252:35309)
0620 08:09:38.269343 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.269345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.269350 (+ 5us) server_negotiation.cc:407] Connection header received
0620 08:09:38.269364 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.269364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.269371 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.269436 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.269919 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.270683 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.271801 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.272093 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.272168 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.273593 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.273595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.273595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.273601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.275546 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.275888 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.275889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.275889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.275933 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.276104 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.276105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.276105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.276224 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.276289 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.276792 (+ 503us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.277069 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":214,"server-negotiator.queue_time_us":101}
I20260620 08:09:38.299505 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19010
I20260620 08:09:38.303112 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.296787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55478)
0620 08:09:38.296833 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.296844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.296898 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.297043 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.297043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.297236 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.297286 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.297287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.297997 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.297998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.298509 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.298511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.298921 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.298922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.299005 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.299473 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.299483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.300228 (+ 745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.301997 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.301998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.301998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.302189 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.302443 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.302444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.302445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.302509 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.302784 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.302786 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.302827 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.302913 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.303018 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 08:09:38.303113 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.296840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.12.132.254:37251)
0620 08:09:38.296912 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.296914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.296924 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:38.296956 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.296957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.296966 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.297039 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.297390 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.297903 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.298607 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.298790 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.298897 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.300320 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.300322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.300322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.300330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.301904 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.302313 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.302314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.302314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.302373 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.302573 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.302574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.302575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.302706 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.302781 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.302939 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.303017 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:09:38.306306 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:40273
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:37251
--webserver_port=35987
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942977, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.254@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
I20260620 08:09:38.311896 19672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:38.311990 19672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } }
I20260620 08:09:38.312678 19672 leader_election.cc:288] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251), c11b170239114a1ebed558dd73006ee4 (127.12.132.253:36483)
I20260620 08:09:38.312963 18958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b66ce99293c4d31949a86e20a77c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da20f1d0b50541c596936809d58f6c31" is_pre_election: true
I20260620 08:09:38.313295 19555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b66ce99293c4d31949a86e20a77c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c11b170239114a1ebed558dd73006ee4" is_pre_election: true
I20260620 08:09:38.313414 19555 raft_consensus.cc:1799] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Handling vote request from an unknown peer 9b66ce99293c4d31949a86e20a77c454
I20260620 08:09:38.313472 19555 raft_consensus.cc:2396] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 9b66ce99293c4d31949a86e20a77c454 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260620 08:09:38.313812 19615 leader_election.cc:302] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b66ce99293c4d31949a86e20a77c454; no voters: c11b170239114a1ebed558dd73006ee4, da20f1d0b50541c596936809d58f6c31
I20260620 08:09:38.314062 19672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:38.314973 19672 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 08:09:38.315963 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.306474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56242)
0620 08:09:38.306533 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.306545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.306592 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.306842 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.306843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.306993 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.307028 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.307029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.307808 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.307809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.308674 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.308677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.309134 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.309135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.309208 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.309594 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.309600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.311146 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.313754 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.313755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.313755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.313975 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.315214 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.315215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.315216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.315289 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.315576 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.315577 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.315629 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.315710 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.315826 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:09:38.316298 19670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.306674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.12.132.252:35309)
0620 08:09:38.306727 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.306729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.306738 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.306770 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.306771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.306780 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.306837 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.307102 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.307708 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.308766 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.309058 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.309125 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.311235 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.311236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.311237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.311244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.313365 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.315079 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.315080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.315080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.315134 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.315375 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.315376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.315376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.315502 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.315570 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.316100 (+ 530us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.316180 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:09:38.316707 18948 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.132.252:35309: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9b66ce99293c4d31949a86e20a77c454"
member_type: NON_VOTER
last_known_addr {
host: "127.12.132.252"
port: 35309
}
attrs {
promote: true
}
}
dest_uuid: "da20f1d0b50541c596936809d58f6c31"
cas_config_opid_index: 5
I20260620 08:09:38.316934 18948 consensus_queue.cc:237] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } }
I20260620 08:09:38.317303 19507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.317807 19641 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 08:09:38.318035 19507 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.132.252:35309
I20260620 08:09:38.318099 19507 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.318171 19507 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:38.318212 19507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:38.318248 19599 master.cc:537] Master@127.12.132.252:35309 shutting down...
W20260620 08:09:38.318936 18932 consensus_peers.cc:487] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 -> Peer 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252:35309): Couldn't send request to peer 9b66ce99293c4d31949a86e20a77c454. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 08:09:38.321885 19599 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:38.321969 19599 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:38.322018 19599 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: stopping tablet replica
I20260620 08:09:38.323475 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.317365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58648)
0620 08:09:38.317400 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.317411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.317452 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.318364 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.318365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.318367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.318407 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.318409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.320739 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.320740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.322444 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.322448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.323228 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.323230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.323297 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.323317 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.323372 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.323408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260620 08:09:38.323894 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.317989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.12.132.253:36483)
0620 08:09:38.318026 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.318028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.318035 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:38.318059 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.318059 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.318077 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.318143 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.318494 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.319219 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.322529 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.323140 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.323222 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.323284 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.323785 (+ 501us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.323830 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:09:38.324095 19555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 08:09:38.324270 19692 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 08:09:38.324785 19555 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.325824 19584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da20f1d0b50541c596936809d58f6c31. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.325906 19584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: LEARNER
I20260620 08:09:38.325944 19584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: NON_VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: true } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.325987 19584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: LEARNER
I20260620 08:09:38.326206 19506 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: attempting to promote NON_VOTER c11b170239114a1ebed558dd73006ee4 to VOTER
I20260620 08:09:38.326391 19506 consensus_queue.cc:237] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } }
I20260620 08:09:38.326781 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.326680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56252)
0620 08:09:38.326720 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.326749 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:35309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 08:09:38.327119 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.327013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56266)
0620 08:09:38.327066 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.327097 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:35309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 08:09:38.327189 18932 consensus_peers.cc:487] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 -> Peer 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252:35309): Couldn't send request to peer 9b66ce99293c4d31949a86e20a77c454. Status: Network error: Client connection negotiation failed: client connection to 127.12.132.252:35309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 08:09:38.327275 19555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 LEARNER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 08:09:38.327433 19692 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 08:09:38.328008 19507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c11b170239114a1ebed558dd73006ee4 (127.12.132.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.329106 19507 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.329181 19507 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:38.329668 19506 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.329845 19506 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:38.330502 19555 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c11b170239114a1ebed558dd73006ee4 (127.12.132.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.331833 19694 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.332010 19694 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:09:38.336422 19599 master.cc:552] Master@127.12.132.252:35309 shutdown complete.
I20260620 08:09:38.336472 19599 master_runner.cc:305] Clearing existing system tablet
I20260620 08:09:38.336632 19599 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.336680 19599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.337383 19599 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:38.337909 19697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.338066 19599 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.338119 19599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:38.338140 19599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "9b66ce99293c4d31949a86e20a77c454"
format_stamp: "Formatted at 2026-06-20 08:09:38 on dist-test-slave-z0sp"
server_key: "2f3b22b5284f4b703e70c7eae619c0e7"
server_key_iv: "e0927d7b0f42adb5ee770ba871c6a727"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.338841 19599 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:38.340284 19599 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 08:09:38.340369 19599 log.cc:1196] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 08:09:38.340687 19599 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Deleting consensus metadata
I20260620 08:09:38.340844 19599 master_runner.cc:315] Copying system tablet from 127.12.132.254:37251
I20260620 08:09:38.341436 19599 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: tablet copy: Beginning tablet copy session from remote peer at address 127.12.132.254:37251
I20260620 08:09:38.348548 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.341721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55484)
0620 08:09:38.341834 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.341844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.341898 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.342049 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.342050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.342254 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.342314 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.342316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.343333 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.343334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.344152 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.344154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.344657 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.344658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.344727 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.345221 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.345227 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.345600 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.347416 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.347417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.347418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.347628 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.347852 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.347852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.347853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.347914 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.348207 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.348208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.348253 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.348323 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.348420 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260620 08:09:38.348582 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.341792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.12.132.254:37251)
0620 08:09:38.341833 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.341836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.341886 (+ 50us) server_negotiation.cc:407] Connection header received
0620 08:09:38.341965 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.341965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.341975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.342045 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.342398 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.343239 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.344251 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.344569 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.344634 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.345699 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.345701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.345701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.345708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.347327 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.347714 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.347715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.347716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.347778 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.347994 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.347996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.347996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.348132 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.348210 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.348372 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.348451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:38.349153 18968 tablet_copy_service.cc:140] P da20f1d0b50541c596936809d58f6c31: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9b66ce99293c4d31949a86e20a77c454 ({username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:55484)
I20260620 08:09:38.349206 18968 tablet_copy_service.cc:161] P da20f1d0b50541c596936809d58f6c31: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9b66ce99293c4d31949a86e20a77c454 at {username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:55484: session id = 9b66ce99293c4d31949a86e20a77c454-00000000000000000000000000000000
I20260620 08:09:38.349818 18968 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 08:09:38.350440 19599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 08:09:38.351675 19599 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: tablet copy: Starting download of 0 data blocks...
I20260620 08:09:38.351799 19599 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: tablet copy: Starting download of 1 WAL segments...
I20260620 08:09:38.352794 19599 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 08:09:38.353909 18968 tablet_copy_service.cc:342] P da20f1d0b50541c596936809d58f6c31: Request end of tablet copy session 9b66ce99293c4d31949a86e20a77c454-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:55484
I20260620 08:09:38.353996 18968 tablet_copy_service.cc:433] P da20f1d0b50541c596936809d58f6c31: ending tablet copy session 9b66ce99293c4d31949a86e20a77c454-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9b66ce99293c4d31949a86e20a77c454
I20260620 08:09:38.357782 19599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:38.359315 19704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.362665 19707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.362962 19599 server_base.cc:1034] running on GCE node
W20260620 08:09:38.363870 19705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.364215 19599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:38.364380 19599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:38.365518 19599 hybrid_clock.cc:648] HybridClock initialized: now 1781942978365509 us; error 27 us; skew 500 ppm
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:38.367030 19599 init.cc:377] Logged in from keytab as oryx/127.12.132.252@KRBTEST.COM (short username oryx)
I20260620 08:09:38.367297 19599 webserver.cc:460] Webserver started at http://127.12.132.252:45211/ using document root <none> and password file <none>
I20260620 08:09:38.367413 19599 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.367447 19599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.368193 19599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.368822 19714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.368995 19599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.369058 19599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "9b66ce99293c4d31949a86e20a77c454"
format_stamp: "Formatted at 2026-06-20 08:09:38 on dist-test-slave-z0sp"
server_key: "2f3b22b5284f4b703e70c7eae619c0e7"
server_key_iv: "e0927d7b0f42adb5ee770ba871c6a727"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.369119 19599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.385741 19599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.387511 19599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.390755 19599 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.252:35309
I20260620 08:09:38.391171 19599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
I20260620 08:09:38.392297 19767 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:09:38.392602 19766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.252:35309 every 8 connection(s)
I20260620 08:09:38.392904 19767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Bootstrap starting.
I20260620 08:09:38.396173 19767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:09:38.396386 19767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Bootstrap complete.
I20260620 08:09:38.396621 19767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } }
I20260620 08:09:38.396709 19767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9b66ce99293c4d31949a86e20a77c454, State: Initialized, Role: LEARNER
I20260620 08:09:38.396747 19767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } }
I20260620 08:09:38.396950 19767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:38.396940 19770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.397190 19770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: LEARNER
I20260620 08:09:38.398707 19781 catalog_manager.cc:1259] Loaded cluster ID: 0de3eb7b4ae340daa18eca021c05787c
I20260620 08:09:38.398873 19781 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: loading cluster ID for follower catalog manager: success
I20260620 08:09:38.400219 19781 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: acquiring CA information for follower catalog manager: success
I20260620 08:09:38.400609 19781 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 08:09:38.412292 19691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:38.412536 19691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:38.412598 19691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:38.413807 19691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:38.413869 19691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:38.413918 19691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:38.415369 19691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:40273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=35987
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:38.415619 19691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:38.415853 19691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:38.420650 19786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.420717 19787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.420954 19789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.421103 19691 server_base.cc:1034] running on GCE node
I20260620 08:09:38.421319 19691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:38.421566 19691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:38.422787 19691 hybrid_clock.cc:648] HybridClock initialized: now 1781942978422776 us; error 26 us; skew 500 ppm
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:38.425361 19691 init.cc:377] Logged in from keytab as oryx/127.12.132.193@KRBTEST.COM (short username oryx)
I20260620 08:09:38.425841 19691 webserver.cc:460] Webserver started at http://127.12.132.193:35987/ using document root <none> and password file <none>
I20260620 08:09:38.426077 19691 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.426132 19691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.427459 19691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.428072 19796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.428227 19691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:38.428305 19691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "28e3bb56273b43a9aa0437c42b9f7e23"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "27e8c69f8d2b4264eb9b1bcf1849d13b"
server_key_iv: "6fceaa1703d6787cfe7e496b498a5803"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.428736 19691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.444775 19691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.447012 19691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:38.447171 19691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.447399 19691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:38.447708 19691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:38.447764 19691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.447811 19691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:38.447840 19691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.456034 19691 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:40273
I20260620 08:09:38.456125 19909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:40273 every 8 connection(s)
I20260620 08:09:38.456656 19691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:38.462849 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19691
I20260620 08:09:38.463013 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19144
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
I20260620 08:09:38.469861 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.457643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:54411 (local address 127.12.132.254:37251)
0620 08:09:38.457722 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.457725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.458128 (+ 403us) server_negotiation.cc:407] Connection header received
0620 08:09:38.459356 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.459358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.459376 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.459447 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.460456 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461285 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462578 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.462846 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462926 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.466266 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.466268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.466268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.466276 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.468030 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468599 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.468600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.468600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.468677 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.469047 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.469048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.469049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.469195 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.469240 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.469602 (+ 362us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.469693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 08:09:38.469988 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.457564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.12.132.193:45189)
0620 08:09:38.457937 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.457987 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.459192 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.459673 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.459674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.460435 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.460544 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.460547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461438 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.461439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.462264 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.462265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.462703 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.462850 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.463777 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.463787 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.466464 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.468484 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.468485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.468743 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.468964 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468964 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.468965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.469019 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.469313 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.469313 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.469454 (+ 141us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.469550 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.469726 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":99,"threads_started":1}
I20260620 08:09:38.470320 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.457645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:45189 (local address 127.12.132.253:36483)
0620 08:09:38.457741 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.457743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.458105 (+ 362us) server_negotiation.cc:407] Connection header received
0620 08:09:38.459552 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.459554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.459568 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.459623 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.460635 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461344 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462375 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.462626 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462688 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.466553 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.466554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.466555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.466563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.468392 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468836 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.468837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.468838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.468895 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.469087 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.469088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.469088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.469231 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.469289 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.470149 (+ 860us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.470237 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 08:09:38.470558 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.457603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:54411)
0620 08:09:38.457942 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.457987 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.459192 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.459562 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.459567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.459967 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.460308 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.460317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461424 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.461425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.462494 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.462496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.462928 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.462998 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.464008 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.464014 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.466152 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.468121 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.468130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.468468 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.468769 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.468770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.468955 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.469284 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.469290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.469453 (+ 163us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.470257 (+ 804us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.470380 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":95,"thread_start_us":97,"threads_started":1}
I20260620 08:09:38.470732 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.458018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:41421 (local address 127.12.132.252:35309)
0620 08:09:38.458346 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.458355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.458365 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:38.459454 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.459456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.459472 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.459554 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.460675 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461321 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.462980 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.463281 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.463338 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.466917 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.466918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.466919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.466926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.468789 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.469147 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.469148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.469148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.469195 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.469393 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.469394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.469394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.469515 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.469570 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.470526 (+ 956us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.470593 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":60,"threads_started":1}
I20260620 08:09:38.470830 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.457907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.12.132.193:41421)
0620 08:09:38.458218 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.458227 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.459306 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.459590 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.459592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.460436 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.460569 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.460571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.461437 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.461438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.462489 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.462492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.463386 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.463387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.463449 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.464093 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.464097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.466798 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.468865 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.468866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.468867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.469075 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.469261 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.469262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.469262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.469327 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.469594 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.469594 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.469630 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.469690 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.469766 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":129,"thread_start_us":70,"threads_started":1}
I20260620 08:09:38.471289 19911 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:38.471403 19911 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.471786 19911 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:38.472448 18948 ts_manager.cc:194] Re-registered known tserver with Master: 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273)
I20260620 08:09:38.472630 19910 heartbeater.cc:347] Connected to a master server at 127.12.132.252:35309
I20260620 08:09:38.472662 19912 heartbeater.cc:347] Connected to a master server at 127.12.132.253:36483
I20260620 08:09:38.472687 19910 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.472694 19912 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.472877 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:44381
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:37251
--webserver_port=44371
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:09:38.472856 19910 heartbeater.cc:511] Master 127.12.132.252:35309 requested a full tablet report, sending...
I20260620 08:09:38.472982 18948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:54411
I20260620 08:09:38.473467 19731 ts_manager.cc:194] Registered new tserver with Master: 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273)
I20260620 08:09:38.472856 19912 heartbeater.cc:511] Master 127.12.132.253:36483 requested a full tablet report, sending...
I20260620 08:09:38.474152 19545 ts_manager.cc:194] Registered new tserver with Master: 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273)
W20260620 08:09:38.560461 19918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:38.560710 19918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:38.560734 19918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:38.561857 19918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:38.561897 19918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:38.561923 19918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:38.563341 19918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:44381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=44371
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:38.563583 19918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:38.563791 19918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:38.568670 19924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.568820 19918 server_base.cc:1034] running on GCE node
W20260620 08:09:38.568653 19926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.568655 19923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.569151 19918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:38.569401 19918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:38.570575 19918 hybrid_clock.cc:648] HybridClock initialized: now 1781942978570552 us; error 55 us; skew 500 ppm
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:38.573107 19918 init.cc:377] Logged in from keytab as oryx/127.12.132.194@KRBTEST.COM (short username oryx)
I20260620 08:09:38.573563 19918 webserver.cc:460] Webserver started at http://127.12.132.194:44371/ using document root <none> and password file <none>
I20260620 08:09:38.573802 19918 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.573861 19918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.575333 19918 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.576110 19933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.576406 19918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:09:38.576485 19918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "fa99f38679a04959aa331e13f05064c8"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "ee60ae7fbc2048d9a63277e8904ee0ea"
server_key_iv: "ac300948316cef99baab87fecc646f17"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.576913 19918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.592624 19918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.594621 19918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:38.594776 19918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.595031 19918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:38.595335 19918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:38.595376 19918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.595396 19918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:38.595409 19918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.601657 19918 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:44381
I20260620 08:09:38.601740 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:44381 every 8 connection(s)
I20260620 08:09:38.602072 19918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:38.608937 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 19918
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:38.609200 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19278
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
I20260620 08:09:38.613233 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.603073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:53439 (local address 127.12.132.254:37251)
0620 08:09:38.603112 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.603114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.603123 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.603653 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.603654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.603665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.603723 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.604510 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.605008 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606396 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.606732 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606787 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.609622 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.609624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.609624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.609631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.611560 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612095 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.612096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.612097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.612154 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612441 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.612442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.612442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.612610 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.612659 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.613042 (+ 383us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.613123 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:09:38.613605 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.602867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:50287 (local address 127.12.132.253:36483)
0620 08:09:38.603017 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.603019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.603029 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:38.603733 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.603734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.603743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.603799 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.604524 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.605264 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606409 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.606708 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606776 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.610671 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.610673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.610673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.610680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.612406 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612798 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.612799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.612799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.612852 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.613051 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.613052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.613052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.613173 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.613214 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.613422 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.613497 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260620 08:09:38.613953 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.602724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.12.132.194:50287)
0620 08:09:38.602972 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.602983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.603599 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.604194 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.604196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.604361 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.604414 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.604415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.605352 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.605353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.606304 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.606307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.606779 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.606915 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.607438 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.607446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.610603 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.612482 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.612483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.612721 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.612923 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.612924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.612979 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.613261 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.613262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.613301 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.613730 (+ 429us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.613815 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":103,"thread_start_us":88,"threads_started":1}
I20260620 08:09:38.613953 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.602724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:53439)
0620 08:09:38.602911 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.602952 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.603545 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.603791 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.603796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.604202 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.604373 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.604379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.605108 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.605108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.606304 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.606307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.607130 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.607131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.607200 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.607568 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.607573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.609499 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.611655 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.611661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.611663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.611988 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.612231 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.612232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.612232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.612362 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.612710 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.612716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.612869 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.613437 (+ 568us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.613595 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":7,"thread_start_us":88,"threads_started":1}
I20260620 08:09:38.614380 20049 heartbeater.cc:347] Connected to a master server at 127.12.132.253:36483
I20260620 08:09:38.614495 20049 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.614679 20049 heartbeater.cc:511] Master 127.12.132.253:36483 requested a full tablet report, sending...
I20260620 08:09:38.615134 19545 ts_manager.cc:194] Registered new tserver with Master: fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
I20260620 08:09:38.615928 20048 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:38.615989 20048 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.616067 20048 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:38.616285 18948 ts_manager.cc:194] Re-registered known tserver with Master: fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
I20260620 08:09:38.616711 18948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:53439
I20260620 08:09:38.617223 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.603229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:46179 (local address 127.12.132.252:35309)
0620 08:09:38.603274 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.603276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.603285 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.603721 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.603723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.603738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.603776 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.604476 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.605258 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.607249 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.607590 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.607687 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.611745 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.611747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.611748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.611756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.613571 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.615822 (+ 2251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.615823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.615824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.615890 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.616098 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.616099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.616099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.616235 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.616291 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.617051 (+ 760us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.617135 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:09:38.617525 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.603067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.12.132.194:46179)
0620 08:09:38.603160 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.603170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.603604 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.603861 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.603862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.604187 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.604373 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.604377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.606436 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.606437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.607175 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.607177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.607691 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.607692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.607761 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.608149 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.608154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.611264 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.613908 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.613909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.613910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.615758 (+ 1848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.615947 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.615948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.615948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.616016 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.616902 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.616903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.616946 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.617013 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.617121 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":64,"threads_started":1}
I20260620 08:09:38.617697 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:33631
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:37251
--webserver_port=40321
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:09:38.618268 20047 heartbeater.cc:347] Connected to a master server at 127.12.132.252:35309
I20260620 08:09:38.618340 20047 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.618477 20047 heartbeater.cc:511] Master 127.12.132.252:35309 requested a full tablet report, sending...
I20260620 08:09:38.618950 19731 ts_manager.cc:194] Registered new tserver with Master: fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
W20260620 08:09:38.698539 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:38.698804 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:38.698844 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:38.699967 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:38.700027 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:38.700073 20054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:38.701568 20054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:33631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=40321
--tserver_master_addrs=127.12.132.254:37251,127.12.132.253:36483,127.12.132.252:35309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:38.701822 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:38.702047 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:38.707139 20060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.707218 20059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:38.707144 20062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:38.707587 20054 server_base.cc:1034] running on GCE node
I20260620 08:09:38.707839 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:38.708050 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:38.709232 20054 hybrid_clock.cc:648] HybridClock initialized: now 1781942978709223 us; error 35 us; skew 500 ppm
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:38.712102 20054 init.cc:377] Logged in from keytab as oryx/127.12.132.195@KRBTEST.COM (short username oryx)
I20260620 08:09:38.712729 20054 webserver.cc:460] Webserver started at http://127.12.132.195:40321/ using document root <none> and password file <none>
I20260620 08:09:38.712966 20054 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:38.713054 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:38.714560 20054 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.715448 20069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.715801 20054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:38.715951 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "70b5607f209c4a03b80b63dd3c663682"
format_stamp: "Formatted at 2026-06-20 08:09:37 on dist-test-slave-z0sp"
server_key: "e8439f1c934260b4e7e4878d1173bbb5"
server_key_iv: "26d95da4faa316cf142b3fbd18e6b749"
server_key_version: "encryptionkey@0"
I20260620 08:09:38.716423 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:38.733076 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:38.735544 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:38.735692 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:38.735924 20054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:38.736240 20054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:38.736292 20054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.736334 20054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:38.736359 20054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:38.742295 20054 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:33631
I20260620 08:09:38.742334 20182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:33631 every 8 connection(s)
I20260620 08:09:38.742668 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:38.743357 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20054
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.253@KRBTEST.COM
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.252@KRBTEST.COM
I20260620 08:09:38.754650 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.744420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.12.132.254:37251)
0620 08:09:38.744743 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.744745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.744754 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.744790 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.744791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.744803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.744876 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.746112 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746789 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.747948 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748460 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748534 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.750220 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.750222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.750222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.750228 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.751952 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.753002 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.753003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.753004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.753061 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.753263 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.753264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.753264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.753409 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.753523 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.754006 (+ 483us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.754527 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":108,"threads_started":1}
I20260620 08:09:38.755019 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:33161 (local address 127.12.132.253:36483)
0620 08:09:38.743808 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.743810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.744125 (+ 315us) server_negotiation.cc:407] Connection header received
0620 08:09:38.744970 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.744971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.744981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.745038 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.745932 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746699 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748284 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748595 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748654 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.751938 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.751939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.751939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.751945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.753433 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.753938 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.753940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.753940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.753998 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.754283 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.754285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.754285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.754419 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.754464 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.754830 (+ 366us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.754935 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:38.755388 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.12.132.195:33161)
0620 08:09:38.744047 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.744056 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.744358 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.745045 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.745046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.745518 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.745829 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.745834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746820 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.746821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748189 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.748192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748706 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748803 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.749451 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.749462 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.751826 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.753520 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.753526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.753528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.753845 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.754071 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.754072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.754073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.754201 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.754510 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.754517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.754676 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.754935 (+ 259us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.755097 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":87,"thread_start_us":55,"threads_started":1}
Jun 20 08:09:38 dist-test-slave-z0sp krb5kdc[18903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942978, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:38.756449 20185 heartbeater.cc:347] Connected to a master server at 127.12.132.253:36483
I20260620 08:09:38.756562 20185 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.756754 20185 heartbeater.cc:511] Master 127.12.132.253:36483 requested a full tablet report, sending...
I20260620 08:09:38.757184 19545 ts_manager.cc:194] Registered new tserver with Master: 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:38.758951 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:34099 (local address 127.12.132.252:35309)
0620 08:09:38.743441 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.743444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.743669 (+ 225us) server_negotiation.cc:407] Connection header received
0620 08:09:38.744461 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.744463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.744474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.744529 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.745935 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746699 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748290 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748623 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748660 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.754080 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.754081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.754082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.754088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.755674 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.756085 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.756086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.756086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.756136 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.756857 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.756858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.756859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.756988 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.758544 (+ 1556us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.758755 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.758831 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:09:38.759334 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.744795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.12.132.252:35309)
0620 08:09:38.748884 (+ 4089us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.748886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.748894 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:38.748914 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.748916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.748927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.748992 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.749862 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.750599 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.751821 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.752146 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.752206 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.753770 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.753772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.753772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.753778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.757712 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.758101 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.758102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.758103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.758160 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.758378 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.758378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.758379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.758514 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.758601 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.758979 (+ 378us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.759253 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4068,"thread_start_us":53,"threads_started":1}
I20260620 08:09:38.757287 18948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 08:09:38.759497 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.12.132.195:34099)
0620 08:09:38.743556 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.743596 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.744349 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.744672 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.744677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.745518 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.745829 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.745834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746820 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.746821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748189 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.748192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748725 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748803 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.749888 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.749893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.752778 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.755774 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.755775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.755775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.756006 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.756211 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.756211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.756212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.756270 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.758595 (+ 2325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.758596 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.758638 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.759070 (+ 432us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.759171 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":97,"thread_start_us":87,"threads_started":1}
W20260620 08:09:38.759536 18948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:38.759667 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:36059 (local address 127.12.132.254:37251)
0620 08:09:38.743988 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.743990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.745440 (+ 1450us) server_negotiation.cc:407] Connection header received
0620 08:09:38.745539 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.745540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.745551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.745582 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.745976 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746765 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748276 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748562 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748597 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.756795 (+ 8198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.756798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.756798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.756805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.758422 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.758791 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.758792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.758792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.758846 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.759160 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.759161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.759161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.759278 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.759322 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.759521 (+ 199us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.759595 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 08:09:38.759790 20183 heartbeater.cc:347] Connected to a master server at 127.12.132.252:35309
I20260620 08:09:38.759825 20183 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.759896 20183 heartbeater.cc:511] Master 127.12.132.252:35309 requested a full tablet report, sending...
I20260620 08:09:38.760036 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.743869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:36059)
0620 08:09:38.745372 (+ 1503us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.745383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.745466 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.745643 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.745644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.745813 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:38.745886 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.745887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.746860 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.746861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748185 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.748187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.748669 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.748670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.748803 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.752207 (+ 3404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:38.752212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:38.756694 (+ 4482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:38.758503 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.758504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.758505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.758700 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.758929 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:38.758929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:38.758930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:38.758982 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:38.759370 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:38.759370 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:38.759409 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.759759 (+ 350us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.759856 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1485,"mutex_wait_us":75,"thread_start_us":64,"threads_started":1}
I20260620 08:09:38.760385 19731 ts_manager.cc:194] Registered new tserver with Master: 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:38.760545 20184 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37251
I20260620 08:09:38.760599 20184 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:38.760663 20184 heartbeater.cc:511] Master 127.12.132.254:37251 requested a full tablet report, sending...
I20260620 08:09:38.760946 18947 ts_manager.cc:194] Re-registered known tserver with Master: 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:38.761354 18947 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:36059
I20260620 08:09:38.763005 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.745305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.12.132.253:36483)
0620 08:09:38.749707 (+ 4402us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.749709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.749728 (+ 19us) server_negotiation.cc:407] Connection header received
0620 08:09:38.749747 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.749748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.749758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.751289 (+ 1531us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.751320 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.752014 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.753099 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.753445 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.753739 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.754812 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.754813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.754814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.754819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.761514 (+ 6695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.761928 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.761928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.761929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.761986 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.762187 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.762187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.762188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.762306 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.762785 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.762824 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.762896 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4384,"thread_start_us":67,"threads_started":1}
I20260620 08:09:38.767894 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57534 (local address 127.12.132.194:44381)
0620 08:09:38.764647 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.764651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.764660 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.764696 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.764698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.764714 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.764911 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.764957 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.765984 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.766886 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767536 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767637 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.767693 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.767784 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.767817 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":75,"threads_started":1}
I20260620 08:09:38.767894 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:44381 (local address 127.0.0.1:57534)
0620 08:09:38.764467 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.764476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.764533 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.764804 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.764805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.764808 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.764844 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.764845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.766077 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.766078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.766781 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.766782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767631 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.767692 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.767697 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.767769 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.767810 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260620 08:09:38.768178 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33631 (local address 127.0.0.1:44122)
0620 08:09:38.764309 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.764316 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.764387 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.764762 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.764763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.764766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.764814 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.764815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.765782 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.765782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.766918 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.766921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767880 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.767945 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.767950 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.767982 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.768027 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":25,"thread_start_us":92,"threads_started":1}
I20260620 08:09:38.768266 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40273 (local address 127.0.0.1:50762)
0620 08:09:38.764092 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.764103 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.764142 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.764647 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.764648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.764650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.764693 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.764695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.766012 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.766013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.767033 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.767035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767880 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.767955 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.767957 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.767987 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.768027 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":5}
I20260620 08:09:38.768397 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.12.132.193:40273)
0620 08:09:38.764429 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.764435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.764447 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:38.764523 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.764526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.764545 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.764763 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.764815 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.765928 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767095 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767789 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.768160 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.768221 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.768247 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.768297 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":157,"threads_started":1}
I20260620 08:09:38.768666 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.764424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.12.132.195:33631)
0620 08:09:38.764596 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.764600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.764610 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:38.764644 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.764646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.764662 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.764880 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.764923 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.765674 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.767116 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.767778 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.768418 (+ 640us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.768527 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.768552 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.768590 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260620 08:09:38.769094 19844 tablet_service.cc:1467] Processing CreateTablet for tablet f76646b38cfd43de8535292c3507e194 (DEFAULT_TABLE table=test-table [id=7d7ce81fc0b34e28b88f15b254cf293f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:38.769250 19981 tablet_service.cc:1467] Processing CreateTablet for tablet f76646b38cfd43de8535292c3507e194 (DEFAULT_TABLE table=test-table [id=7d7ce81fc0b34e28b88f15b254cf293f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:38.769554 19844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76646b38cfd43de8535292c3507e194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:38.769554 19981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76646b38cfd43de8535292c3507e194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:38.769485 20117 tablet_service.cc:1467] Processing CreateTablet for tablet f76646b38cfd43de8535292c3507e194 (DEFAULT_TABLE table=test-table [id=7d7ce81fc0b34e28b88f15b254cf293f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:38.769804 20117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f76646b38cfd43de8535292c3507e194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:38.773054 20210 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Bootstrap starting.
I20260620 08:09:38.773289 20211 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Bootstrap starting.
I20260620 08:09:38.773711 20210 tablet_bootstrap.cc:654] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:38.773833 20211 tablet_bootstrap.cc:654] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:38.774055 20210 log.cc:824] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:38.774091 20211 log.cc:824] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:38.774372 20209 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Bootstrap starting.
I20260620 08:09:38.774793 20211 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: No bootstrap required, opened a new log
I20260620 08:09:38.774873 20211 ts_tablet_manager.cc:1397] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:38.774905 20209 tablet_bootstrap.cc:654] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:38.775252 20209 log.cc:824] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:38.775933 20210 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: No bootstrap required, opened a new log
I20260620 08:09:38.776082 20210 ts_tablet_manager.cc:1397] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:09:38.777376 20211 raft_consensus.cc:348] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.777541 20211 raft_consensus.cc:374] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:38.777587 20211 raft_consensus.cc:729] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b5607f209c4a03b80b63dd3c663682, State: Initialized, Role: FOLLOWER
I20260620 08:09:38.777704 20211 consensus_queue.cc:260] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.778005 20211 ts_tablet_manager.cc:1428] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:09:38.777926 20210 raft_consensus.cc:348] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.778091 20210 raft_consensus.cc:374] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:38.778126 20210 raft_consensus.cc:729] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e3bb56273b43a9aa0437c42b9f7e23, State: Initialized, Role: FOLLOWER
I20260620 08:09:38.778260 20184 heartbeater.cc:503] Master 127.12.132.254:37251 was elected leader, sending a full tablet report...
I20260620 08:09:38.778224 20210 consensus_queue.cc:260] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.778923 20210 ts_tablet_manager.cc:1428] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:09:38.779371 20209 tablet_bootstrap.cc:492] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: No bootstrap required, opened a new log
I20260620 08:09:38.779507 19911 heartbeater.cc:503] Master 127.12.132.254:37251 was elected leader, sending a full tablet report...
I20260620 08:09:38.779600 20209 ts_tablet_manager.cc:1397] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 08:09:38.781756 20209 raft_consensus.cc:348] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.781885 20209 raft_consensus.cc:374] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:38.781906 20209 raft_consensus.cc:729] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa99f38679a04959aa331e13f05064c8, State: Initialized, Role: FOLLOWER
I20260620 08:09:38.782011 20209 consensus_queue.cc:260] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.782260 20209 ts_tablet_manager.cc:1428] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:09:38.782456 20048 heartbeater.cc:503] Master 127.12.132.254:37251 was elected leader, sending a full tablet report...
I20260620 08:09:38.846292 20215 raft_consensus.cc:482] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:38.846455 20215 raft_consensus.cc:504] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.846817 20215 leader_election.cc:288] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273), fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
I20260620 08:09:38.849779 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.847027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:44381 (local address 127.12.132.195:48175)
0620 08:09:38.847073 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.847085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.847126 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.847473 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.847474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.847476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.847526 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.847527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.848177 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.848178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.848951 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.848953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.849490 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.849575 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.849579 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.849629 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.849658 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 08:09:38.849779 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.847117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:48175 (local address 127.12.132.194:44381)
0620 08:09:38.847161 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.847163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.847172 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.847200 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.847201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.847211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.847283 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.847614 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.848092 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849053 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.849408 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849463 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.849470 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.849659 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.849706 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:09:38.849848 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.847118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60727 (local address 127.12.132.193:40273)
0620 08:09:38.847161 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.847163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.847337 (+ 174us) server_negotiation.cc:407] Connection header received
0620 08:09:38.847365 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.847366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.847384 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.847428 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.847616 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.848107 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849088 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.849470 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849543 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.849550 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.849720 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.849763 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:38.849875 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.847036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40273 (local address 127.12.132.195:60727)
0620 08:09:38.847212 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.847224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.847272 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.847449 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.847450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.847451 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.847521 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.847523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.848212 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.848213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.848974 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.848976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.849554 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.849555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.849635 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.849639 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.849697 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.849731 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":165}
I20260620 08:09:38.850162 20001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76646b38cfd43de8535292c3507e194" candidate_uuid: "70b5607f209c4a03b80b63dd3c663682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa99f38679a04959aa331e13f05064c8" is_pre_election: true
I20260620 08:09:38.850176 19864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76646b38cfd43de8535292c3507e194" candidate_uuid: "70b5607f209c4a03b80b63dd3c663682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" is_pre_election: true
I20260620 08:09:38.850328 20001 raft_consensus.cc:2454] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70b5607f209c4a03b80b63dd3c663682 in term 0.
I20260620 08:09:38.850333 19864 raft_consensus.cc:2454] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70b5607f209c4a03b80b63dd3c663682 in term 0.
I20260620 08:09:38.850541 20072 leader_election.cc:302] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70b5607f209c4a03b80b63dd3c663682, fa99f38679a04959aa331e13f05064c8; no voters:
I20260620 08:09:38.850673 20215 raft_consensus.cc:2781] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:38.850726 20215 raft_consensus.cc:482] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:38.850756 20215 raft_consensus.cc:3037] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:38.851456 20215 raft_consensus.cc:504] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.851596 20215 leader_election.cc:288] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [CANDIDATE]: Term 1 election: Requested vote from peers 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273), fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381)
I20260620 08:09:38.851858 19864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76646b38cfd43de8535292c3507e194" candidate_uuid: "70b5607f209c4a03b80b63dd3c663682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28e3bb56273b43a9aa0437c42b9f7e23"
I20260620 08:09:38.851888 20001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f76646b38cfd43de8535292c3507e194" candidate_uuid: "70b5607f209c4a03b80b63dd3c663682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa99f38679a04959aa331e13f05064c8"
I20260620 08:09:38.852025 19864 raft_consensus.cc:3037] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:38.852056 20001 raft_consensus.cc:3037] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:38.853260 20001 raft_consensus.cc:2454] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70b5607f209c4a03b80b63dd3c663682 in term 1.
I20260620 08:09:38.853260 19864 raft_consensus.cc:2454] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70b5607f209c4a03b80b63dd3c663682 in term 1.
W20260620 08:09:38.853500 20050 tablet.cc:2369] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:38.853529 20071 leader_election.cc:302] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e3bb56273b43a9aa0437c42b9f7e23, 70b5607f209c4a03b80b63dd3c663682; no voters:
I20260620 08:09:38.853624 20215 raft_consensus.cc:2781] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:38.853794 20215 raft_consensus.cc:686] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 1 LEADER]: Becoming Leader. State: Replica: 70b5607f209c4a03b80b63dd3c663682, State: Running, Role: LEADER
I20260620 08:09:38.854036 20215 consensus_queue.cc:237] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } }
I20260620 08:09:38.854919 18947 catalog_manager.cc:5495] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "70b5607f209c4a03b80b63dd3c663682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:38.861619 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.858980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:35309 (local address 127.0.0.1:56282)
0620 08:09:38.859039 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:38.859050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:38.859096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:38.859290 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:38.859291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:38.859292 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:38.859350 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.859352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.860034 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.860035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.860809 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:38.860811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.861385 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.861386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:38.861446 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.861450 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:38.861507 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 08:09:38.861547 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 08:09:38.861619 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.859094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.12.132.252:35309)
0620 08:09:38.859151 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.859153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.859162 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.859187 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.859188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.859208 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.859288 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:38.859414 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.859933 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.860899 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.861294 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.861377 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.861448 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.861514 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.861551 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 08:09:38.862356 19741 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260620 08:09:38.863380 19770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da20f1d0b50541c596936809d58f6c31. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.863495 19770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: LEARNER
I20260620 08:09:38.866420 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.864218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.12.132.195:33631)
0620 08:09:38.864260 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.864262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.864271 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:38.864354 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.864355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.864370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.864425 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:38.864572 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.865025 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.865714 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.865907 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.865973 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.866114 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:38.866207 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:38.866266 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.866299 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.866315 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:38.868949 20001 raft_consensus.cc:1261] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Refusing update from remote peer 70b5607f209c4a03b80b63dd3c663682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:38.868999 19864 raft_consensus.cc:1261] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Refusing update from remote peer 70b5607f209c4a03b80b63dd3c663682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:38.869334 20215 consensus_queue.cc:1035] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:38.869482 20218 consensus_queue.cc:1035] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:38.873348 20226 mvcc.cc:205] Tried to move back new op lower bound from 7298838441443602432 to 7298838441386995712. Current Snapshot: MvccSnapshot[applied={T|T < 7298838441443602432 or (T in {7298838441443602432})}]
I20260620 08:09:38.873950 20225 mvcc.cc:205] Tried to move back new op lower bound from 7298838441443602432 to 7298838441386995712. Current Snapshot: MvccSnapshot[applied={T|T < 7298838441443602432}]
I20260620 08:09:38.876296 18948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:38.876396 18948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:38.876956 19692 sys_catalog.cc:455] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da20f1d0b50541c596936809d58f6c31" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } }
I20260620 08:09:38.877045 19692 sys_catalog.cc:458] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:38.877628 19507 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: attempting to promote NON_VOTER 9b66ce99293c4d31949a86e20a77c454 to VOTER
I20260620 08:09:38.877750 19507 consensus_queue.cc:237] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } }
I20260620 08:09:38.878095 18948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31: Sending DeleteTablet for 3 replicas of tablet f76646b38cfd43de8535292c3507e194
I20260620 08:09:38.878393 19844 tablet_service.cc:1514] Processing DeleteTablet for tablet f76646b38cfd43de8535292c3507e194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:38 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:50762
I20260620 08:09:38.878403 19981 tablet_service.cc:1514] Processing DeleteTablet for tablet f76646b38cfd43de8535292c3507e194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:38 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:57534
I20260620 08:09:38.878537 20117 tablet_service.cc:1514] Processing DeleteTablet for tablet f76646b38cfd43de8535292c3507e194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:38 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44122
I20260620 08:09:38.879001 19741 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 LEARNER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260620 08:09:38.878737 20236 tablet_replica.cc:330] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: stopping tablet replica
I20260620 08:09:38.879245 19555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Refusing update from remote peer da20f1d0b50541c596936809d58f6c31: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260620 08:09:38.879429 19692 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260620 08:09:38.879755 20237 tablet_replica.cc:330] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: stopping tablet replica
I20260620 08:09:38.879901 20237 raft_consensus.cc:2229] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:38.880057 20237 raft_consensus.cc:2258] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:38.880101 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 18916
I20260620 08:09:38.880096 19692 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da20f1d0b50541c596936809d58f6c31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260620 08:09:38.880571 20235 tablet_replica.cc:330] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: stopping tablet replica
I20260620 08:09:38.880674 20235 raft_consensus.cc:2229] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:38.880776 20235 raft_consensus.cc:2258] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:38.881237 20235 ts_tablet_manager.cc:1905] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:38.881404 20237 ts_tablet_manager.cc:1905] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:38.883105 20235 ts_tablet_manager.cc:1918] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:38.883185 20235 log.cc:1196] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/f76646b38cfd43de8535292c3507e194
I20260620 08:09:38.879237 20236 raft_consensus.cc:2229] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:38.883322 20237 ts_tablet_manager.cc:1918] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:38.883391 20237 log.cc:1196] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/f76646b38cfd43de8535292c3507e194
I20260620 08:09:38.883472 20235 ts_tablet_manager.cc:1939] T f76646b38cfd43de8535292c3507e194 P 28e3bb56273b43a9aa0437c42b9f7e23: Deleting consensus metadata
I20260620 08:09:38.883467 20236 raft_consensus.cc:2258] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:38.883667 20237 ts_tablet_manager.cc:1939] T f76646b38cfd43de8535292c3507e194 P 70b5607f209c4a03b80b63dd3c663682: Deleting consensus metadata
I20260620 08:09:38.884130 20236 ts_tablet_manager.cc:1905] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:38.885844 20236 ts_tablet_manager.cc:1918] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:38.885989 20236 log.cc:1196] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/f76646b38cfd43de8535292c3507e194
I20260620 08:09:38.886267 20236 ts_tablet_manager.cc:1939] T f76646b38cfd43de8535292c3507e194 P fa99f38679a04959aa331e13f05064c8: Deleting consensus metadata
W20260620 08:09:38.889590 19935 connection.cc:535] server connection from 127.0.0.1:57534 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 08:09:38.889567 19715 connection.cc:535] server connection from 127.0.0.1:56282 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 08:09:38.889880 20073 connection.cc:535] server connection from 127.0.0.1:44122 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 08:09:38.890202 19799 connection.cc:535] server connection from 127.0.0.1:50762 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:09:38.890214 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.879701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.12.132.252:35309)
0620 08:09:38.879754 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.879757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.881093 (+ 1336us) server_negotiation.cc:407] Connection header received
0620 08:09:38.881195 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.881196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.881207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.881234 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.882097 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.882850 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.884120 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.884368 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.884451 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.886416 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.886417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.886418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.886425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.888281 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.889018 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.889020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.889021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.889138 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.889438 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.889439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.889440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.889619 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.889761 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.889998 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.890113 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 08:09:38.901657 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.892619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.12.132.252:35309)
0620 08:09:38.892676 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.892678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.892793 (+ 115us) server_negotiation.cc:407] Connection header received
0620 08:09:38.892851 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.892852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.892861 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.892920 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.893539 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.894193 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.895534 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.895859 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.895932 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.898335 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.898337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.898337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.898344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.900119 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.900584 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.900585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.900586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.900641 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.900992 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.900993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.900993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.901155 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.901263 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.901465 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.901555 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:09:38.902041 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:38.892306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.12.132.253:36483)
0620 08:09:38.892374 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:38.892377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:38.892547 (+ 170us) server_negotiation.cc:407] Connection header received
0620 08:09:38.892662 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:38.892663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:38.892680 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:38.892742 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:38.894076 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.894870 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.895929 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:38.896266 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:38.896375 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:38.898284 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:38.898285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:38.898286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:38.898293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:38.900392 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.900920 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.900921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.900922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.900991 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:38.901213 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:38.901214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:38.901214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:38.901346 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:38.901444 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:38.901589 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 08:09:38.901678 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:09:39.874470 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:39.874151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:37403)
0620 08:09:39.874314 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:39.874399 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260620 08:09:39.874588 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:39.874142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:58445)
0620 08:09:39.874447 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:39.874537 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":75,"threads_started":1}
W20260620 08:09:39.874679 19911 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
W20260620 08:09:39.874763 20184 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:39.874850 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:39.874166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:33545)
0620 08:09:39.874536 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:39.874803 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":51,"threads_started":1}
W20260620 08:09:39.875070 20048 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37251 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:40.318109 20249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da20f1d0b50541c596936809d58f6c31)
I20260620 08:09:40.318225 20249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } }
I20260620 08:09:40.318431 20249 leader_election.cc:288] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251), c11b170239114a1ebed558dd73006ee4 (127.12.132.253:36483)
I20260620 08:09:40.318852 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.318601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55962)
0620 08:09:40.318754 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.318821 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260620 08:09:40.319027 19717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:40.319453 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.319361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55968)
0620 08:09:40.319409 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.319435 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 08:09:40.319562 19717 leader_election.cc:334] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251): Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:40.321769 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.318719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.12.132.253:36483)
0620 08:09:40.318840 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:40.318842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:40.318854 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:40.318884 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:40.318885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:40.318896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:40.319011 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:40.319194 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:40.319898 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:40.320959 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:40.321326 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:40.321390 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:40.321397 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:40.321686 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 08:09:40.321718 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260620 08:09:40.321877 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.318627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:36483 (local address 127.0.0.1:58664)
0620 08:09:40.318758 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.318770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:40.318829 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:40.319040 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:40.319042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:40.319044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:40.319106 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:40.319109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:40.319978 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:40.319978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:40.320868 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:40.320870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:40.321473 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:40.321475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:40.321558 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:40.321575 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 08:09:40.321668 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 08:09:40.321735 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260620 08:09:40.322243 19555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b66ce99293c4d31949a86e20a77c454" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c11b170239114a1ebed558dd73006ee4" is_pre_election: true
I20260620 08:09:40.322525 19715 leader_election.cc:302] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b66ce99293c4d31949a86e20a77c454; no voters: c11b170239114a1ebed558dd73006ee4, da20f1d0b50541c596936809d58f6c31
I20260620 08:09:40.322666 20249 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:09:40.379132 20254 raft_consensus.cc:482] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da20f1d0b50541c596936809d58f6c31)
I20260620 08:09:40.379412 20254 raft_consensus.cc:504] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } }
I20260620 08:09:40.379755 20254 leader_election.cc:288] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251)
I20260620 08:09:40.380383 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.379991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:55994)
0620 08:09:40.380245 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.380332 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":71,"threads_started":1}
W20260620 08:09:40.380617 19531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:40.381100 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.381000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56004)
0620 08:09:40.381050 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.381082 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 08:09:40.381228 19531 leader_election.cc:334] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251): Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:40.381296 19531 leader_election.cc:302] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: c11b170239114a1ebed558dd73006ee4; no voters: da20f1d0b50541c596936809d58f6c31
I20260620 08:09:40.381404 20254 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:09:40.875553 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.875106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:42931)
0620 08:09:40.875450 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.875494 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":64,"threads_started":1}
I20260620 08:09:40.875842 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.875111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:43941)
0620 08:09:40.875721 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.875772 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":82,"threads_started":1}
I20260620 08:09:40.875957 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:40.875604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:34715)
0620 08:09:40.875872 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:40.875905 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":55,"threads_started":1}
I20260620 08:09:41.876401 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:41.875972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:37329)
0620 08:09:41.876306 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:41.876356 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":66,"threads_started":1}
I20260620 08:09:41.876713 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:41.876356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:51627)
0620 08:09:41.876631 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:41.876668 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":42,"threads_started":1}
I20260620 08:09:41.876744 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:41.876322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:35171)
0620 08:09:41.876673 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:41.876702 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":71,"threads_started":1}
I20260620 08:09:42.034888 20263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da20f1d0b50541c596936809d58f6c31)
I20260620 08:09:42.035073 20263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } }
I20260620 08:09:42.035303 20263 leader_election.cc:288] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251), c11b170239114a1ebed558dd73006ee4 (127.12.132.253:36483)
I20260620 08:09:42.035588 19555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b66ce99293c4d31949a86e20a77c454" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c11b170239114a1ebed558dd73006ee4" is_pre_election: true
I20260620 08:09:42.035632 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.035446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56278)
0620 08:09:42.035548 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.035587 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260620 08:09:42.035707 19555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b66ce99293c4d31949a86e20a77c454 in term 1.
I20260620 08:09:42.035861 19715 leader_election.cc:302] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b66ce99293c4d31949a86e20a77c454, c11b170239114a1ebed558dd73006ee4; no voters:
I20260620 08:09:42.036024 20263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 08:09:42.036067 20263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da20f1d0b50541c596936809d58f6c31)
I20260620 08:09:42.036082 20263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:09:42.036254 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.035931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56286)
0620 08:09:42.036193 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.036229 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
W20260620 08:09:42.036365 19717 leader_election.cc:334] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251): Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:42.036993 20263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } }
I20260620 08:09:42.037135 20263 leader_election.cc:288] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 election: Requested vote from peers da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251), c11b170239114a1ebed558dd73006ee4 (127.12.132.253:36483)
I20260620 08:09:42.037307 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.037210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56300)
0620 08:09:42.037247 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.037278 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 08:09:42.037283 19555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b66ce99293c4d31949a86e20a77c454" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c11b170239114a1ebed558dd73006ee4"
I20260620 08:09:42.037351 19555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:09:42.037627 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.037522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56306)
0620 08:09:42.037578 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.037604 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 08:09:42.037711 19717 leader_election.cc:334] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251): Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
I20260620 08:09:42.038167 19555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b66ce99293c4d31949a86e20a77c454 in term 2.
I20260620 08:09:42.038349 19715 leader_election.cc:302] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b66ce99293c4d31949a86e20a77c454, c11b170239114a1ebed558dd73006ee4; no voters: da20f1d0b50541c596936809d58f6c31
I20260620 08:09:42.038432 20263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:09:42.038539 20263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 2 LEADER]: Becoming Leader. State: Replica: 9b66ce99293c4d31949a86e20a77c454, State: Running, Role: LEADER
I20260620 08:09:42.038643 20263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } }
I20260620 08:09:42.039016 20266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b66ce99293c4d31949a86e20a77c454. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.039126 20266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:42.039312 20268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:42.039700 20268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:42.039763 20268 catalog_manager.cc:1259] Loaded cluster ID: 0de3eb7b4ae340daa18eca021c05787c
I20260620 08:09:42.039779 20268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:42.039909 20268 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:42.039958 20268 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Loaded TSK: 0
I20260620 08:09:42.040062 20268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:09:42.070741 19729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:42.072283 19729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:42.073412 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.073291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56328)
0620 08:09:42.073345 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.073379 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:09:42.073410 19555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 2 FOLLOWER]: Refusing update from remote peer 9b66ce99293c4d31949a86e20a77c454: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260620 08:09:42.073647 20266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260620 08:09:42.073761 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.073661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.0.0.1:56344)
0620 08:09:42.073713 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.073741 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 08:09:42.073858 19717 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 -> Peer da20f1d0b50541c596936809d58f6c31 (127.12.132.254:37251): Couldn't send request to peer da20f1d0b50541c596936809d58f6c31. Status: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 08:09:42.074272 20270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b66ce99293c4d31949a86e20a77c454. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: NON_VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.074481 20270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:09:42.074651 20266 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [term 2 LEADER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.074745 19555 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [term 2 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 9b66ce99293c4d31949a86e20a77c454 (127.12.132.252) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.075872 20269 mvcc.cc:205] Tried to move back new op lower bound from 7298838454570659840 to 7298838454430916608. Current Snapshot: MvccSnapshot[applied={T|T < 7298838454570659840}]
I20260620 08:09:42.075830 20266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.075994 20266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:42.075948 20270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.075824 20263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.076035 20270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:09:42.076076 20263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:42.076074 20270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9b66ce99293c4d31949a86e20a77c454" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "da20f1d0b50541c596936809d58f6c31" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37251 } } peers { permanent_uuid: "c11b170239114a1ebed558dd73006ee4" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 36483 } attrs { promote: false } } peers { permanent_uuid: "9b66ce99293c4d31949a86e20a77c454" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 35309 } attrs { promote: false } } }
I20260620 08:09:42.076116 20271 mvcc.cc:205] Tried to move back new op lower bound from 7298838454570659840 to 7298838454430916608. Current Snapshot: MvccSnapshot[applied={T|T < 7298838454570659840}]
I20260620 08:09:42.076138 20270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11b170239114a1ebed558dd73006ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:09:42.081441 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.078239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.12.132.193:40273)
0620 08:09:42.078345 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.078347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.078430 (+ 83us) server_negotiation.cc:407] Connection header received
0620 08:09:42.078459 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.078460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.078472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.078535 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:42.078724 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079331 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.080734 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.081128 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.081185 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.081192 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.081328 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.081352 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":47,"threads_started":1}
I20260620 08:09:42.081458 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.077995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40273 (local address 127.0.0.1:50778)
0620 08:09:42.078343 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.078349 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:42.078384 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:42.078591 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:42.078592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:42.078594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:42.078662 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.078664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079407 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.079408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.080642 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.080644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.081200 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.081201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.081259 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.081263 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:42.081323 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 08:09:42.081369 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":3,"thread_start_us":157,"threads_started":1}
I20260620 08:09:42.081640 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.077947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:44381 (local address 127.0.0.1:57538)
0620 08:09:42.078211 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.078222 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:42.078279 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:42.078625 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:42.078625 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:42.078628 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:42.078669 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.078670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079609 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.079609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.080704 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.080706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.081409 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.081410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.081474 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.081478 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:42.081535 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 08:09:42.081577 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":30,"thread_start_us":169,"threads_started":1}
I20260620 08:09:42.081808 19844 tablet_service.cc:1467] Processing CreateTablet for tablet e1f7f997b618456e92862ff117b33261 (DEFAULT_TABLE table=test-table [id=2c454ab7b3f2424e8d83bbc01cb69629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:42.081852 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.078042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.12.132.194:44381)
0620 08:09:42.078446 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.078448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.078454 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:09:42.078500 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.078501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.078512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.078577 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:42.078746 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079544 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.080777 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.081331 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.081404 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.081412 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.081725 (+ 313us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.081776 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":120,"threads_started":1}
I20260620 08:09:42.081996 19844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f7f997b618456e92862ff117b33261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:42.082078 19981 tablet_service.cc:1467] Processing CreateTablet for tablet e1f7f997b618456e92862ff117b33261 (DEFAULT_TABLE table=test-table [id=2c454ab7b3f2424e8d83bbc01cb69629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:42.082192 19981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f7f997b618456e92862ff117b33261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:42.083082 20282 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23: Bootstrap starting.
I20260620 08:09:42.083537 20282 tablet_bootstrap.cc:654] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:42.083583 20283 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Bootstrap starting.
I20260620 08:09:42.083714 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.077947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33631 (local address 127.0.0.1:44144)
0620 08:09:42.077999 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.078010 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:42.078061 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:42.078904 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:42.078905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:42.078906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:42.078939 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.078940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079775 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.079776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.080917 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.080919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.083504 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.083505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.083564 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.083567 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:42.083615 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 08:09:42.083659 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:09:42.083941 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.078131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.12.132.195:33631)
0620 08:09:42.078775 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.078777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.078784 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:42.078808 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.078808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.078819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.078887 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:42.079013 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.079695 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.081444 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.083437 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.083507 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.083514 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.083812 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.083872 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":49,"threads_started":1}
I20260620 08:09:42.084096 20283 tablet_bootstrap.cc:654] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:42.084188 20117 tablet_service.cc:1467] Processing CreateTablet for tablet e1f7f997b618456e92862ff117b33261 (DEFAULT_TABLE table=test-table [id=2c454ab7b3f2424e8d83bbc01cb69629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:42.084300 20117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f7f997b618456e92862ff117b33261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:42.084685 20283 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: No bootstrap required, opened a new log
I20260620 08:09:42.084738 20283 ts_tablet_manager.cc:1397] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:42.084793 20282 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23: No bootstrap required, opened a new log
I20260620 08:09:42.084828 20282 ts_tablet_manager.cc:1397] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:42.084846 20283 raft_consensus.cc:348] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.084921 20283 raft_consensus.cc:374] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:42.084941 20283 raft_consensus.cc:729] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa99f38679a04959aa331e13f05064c8, State: Initialized, Role: FOLLOWER
I20260620 08:09:42.084983 20282 raft_consensus.cc:348] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.085052 20282 raft_consensus.cc:374] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:42.085069 20282 raft_consensus.cc:729] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e3bb56273b43a9aa0437c42b9f7e23, State: Initialized, Role: FOLLOWER
I20260620 08:09:42.085111 20282 consensus_queue.cc:260] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.085211 20283 consensus_queue.cc:260] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.085294 20282 ts_tablet_manager.cc:1428] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:42.085616 20283 ts_tablet_manager.cc:1428] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 08:09:42.085635 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.085536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:43371)
0620 08:09:42.085579 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.085606 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 08:09:42.085992 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.085895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:55497)
0620 08:09:42.085929 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.085963 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260620 08:09:42.086040 19910 heartbeater.cc:503] Master 127.12.132.252:35309 was elected leader, sending a full tablet report...
I20260620 08:09:42.086346 20287 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Bootstrap starting.
I20260620 08:09:42.086535 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.086443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:40547)
0620 08:09:42.086482 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.086507 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 08:09:42.086714 20047 heartbeater.cc:503] Master 127.12.132.252:35309 was elected leader, sending a full tablet report...
I20260620 08:09:42.087006 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.086894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:34209)
0620 08:09:42.086943 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.086978 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 08:09:42.087016 20287 tablet_bootstrap.cc:654] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:42.087546 20287 tablet_bootstrap.cc:492] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: No bootstrap required, opened a new log
I20260620 08:09:42.087610 20287 ts_tablet_manager.cc:1397] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:42.087713 20287 raft_consensus.cc:348] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.087792 20287 raft_consensus.cc:374] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:42.087818 20287 raft_consensus.cc:729] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b5607f209c4a03b80b63dd3c663682, State: Initialized, Role: FOLLOWER
I20260620 08:09:42.087872 20287 consensus_queue.cc:260] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.088039 20287 ts_tablet_manager.cc:1428] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:42.088672 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.088559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:36939)
0620 08:09:42.088609 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.088646 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 08:09:42.088680 20183 heartbeater.cc:503] Master 127.12.132.252:35309 was elected leader, sending a full tablet report...
I20260620 08:09:42.143358 20286 raft_consensus.cc:482] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:42.143548 20286 raft_consensus.cc:504] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.144028 20286 leader_election.cc:288] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381), 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:42.148872 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.144249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:44381 (local address 127.12.132.193:54895)
0620 08:09:42.144299 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.144322 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:42.144381 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:42.144716 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:42.144716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:42.144719 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:42.144814 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.144815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.145995 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.146002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.147644 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.147645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.148618 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.148619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.148689 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.148694 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:42.148764 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 08:09:42.148806 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":38}
I20260620 08:09:42.149135 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.144391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:46259 (local address 127.12.132.195:33631)
0620 08:09:42.144522 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.144524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.144780 (+ 256us) server_negotiation.cc:407] Connection header received
0620 08:09:42.144816 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.144817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.144825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.144909 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:42.145011 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.145838 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.147785 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.148536 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.148616 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.148632 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.148963 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.149049 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 08:09:42.149170 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.144249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33631 (local address 127.12.132.193:46259)
0620 08:09:42.144683 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.144694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:42.144755 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:42.144911 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:42.144912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:42.144913 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:42.144959 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.144960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.145995 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.145997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.147641 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:42.147645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.148717 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.148719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:42.148836 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.148842 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 08:09:42.148913 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 08:09:42.148971 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":296,"thread_start_us":171,"threads_started":1}
I20260620 08:09:42.148958 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.144371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:54895 (local address 127.12.132.194:44381)
0620 08:09:42.144464 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.144467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.144483 (+ 16us) server_negotiation.cc:407] Connection header received
0620 08:09:42.144590 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.144591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.144601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.144689 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:42.144907 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.145851 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.147785 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.148454 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.148625 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.148637 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.148798 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.148856 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 08:09:42.149426 20137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f7f997b618456e92862ff117b33261" candidate_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b5607f209c4a03b80b63dd3c663682" is_pre_election: true
I20260620 08:09:42.149519 20001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f7f997b618456e92862ff117b33261" candidate_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa99f38679a04959aa331e13f05064c8" is_pre_election: true
I20260620 08:09:42.149609 20001 raft_consensus.cc:2454] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e3bb56273b43a9aa0437c42b9f7e23 in term 0.
I20260620 08:09:42.149606 20137 raft_consensus.cc:2454] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28e3bb56273b43a9aa0437c42b9f7e23 in term 0.
I20260620 08:09:42.149894 19800 leader_election.cc:302] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e3bb56273b43a9aa0437c42b9f7e23, 70b5607f209c4a03b80b63dd3c663682; no voters:
I20260620 08:09:42.150157 20286 raft_consensus.cc:2781] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:42.150220 20286 raft_consensus.cc:482] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:42.150247 20286 raft_consensus.cc:3037] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:42.150976 20286 raft_consensus.cc:504] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.151144 20286 leader_election.cc:288] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [CANDIDATE]: Term 1 election: Requested vote from peers fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381), 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631)
I20260620 08:09:42.151309 20137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f7f997b618456e92862ff117b33261" candidate_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b5607f209c4a03b80b63dd3c663682"
I20260620 08:09:42.151311 20001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f7f997b618456e92862ff117b33261" candidate_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa99f38679a04959aa331e13f05064c8"
I20260620 08:09:42.151386 20137 raft_consensus.cc:3037] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:42.151386 20001 raft_consensus.cc:3037] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:42.152160 20137 raft_consensus.cc:2454] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e3bb56273b43a9aa0437c42b9f7e23 in term 1.
I20260620 08:09:42.152160 20001 raft_consensus.cc:2454] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28e3bb56273b43a9aa0437c42b9f7e23 in term 1.
I20260620 08:09:42.152310 19799 leader_election.cc:302] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28e3bb56273b43a9aa0437c42b9f7e23, fa99f38679a04959aa331e13f05064c8; no voters:
I20260620 08:09:42.152448 20286 raft_consensus.cc:2781] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:42.152671 20286 raft_consensus.cc:686] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [term 1 LEADER]: Becoming Leader. State: Replica: 28e3bb56273b43a9aa0437c42b9f7e23, State: Running, Role: LEADER
I20260620 08:09:42.152781 20286 consensus_queue.cc:237] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } }
I20260620 08:09:42.153329 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.153231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:60291)
0620 08:09:42.153272 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.153299 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 08:09:42.153496 19730 catalog_manager.cc:5495] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28e3bb56273b43a9aa0437c42b9f7e23" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40273 } health_report { overall_health: HEALTHY } } }
I20260620 08:09:42.176445 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.173577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.12.132.193:40273)
0620 08:09:42.173651 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:42.173657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:42.173738 (+ 81us) server_negotiation.cc:407] Connection header received
0620 08:09:42.173794 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:42.173796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:42.173821 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:42.173931 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:42.174210 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.174697 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.175640 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:42.175840 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:42.175909 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:42.176097 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:42.176199 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:42.176257 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:42.176372 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 08:09:42.176391 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:09:42.177889 20001 raft_consensus.cc:1261] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Refusing update from remote peer 28e3bb56273b43a9aa0437c42b9f7e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:42.177929 20137 raft_consensus.cc:1261] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Refusing update from remote peer 28e3bb56273b43a9aa0437c42b9f7e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:42.178339 20292 consensus_queue.cc:1035] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa99f38679a04959aa331e13f05064c8" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:42.178560 20286 consensus_queue.cc:1035] T e1f7f997b618456e92862ff117b33261 P 28e3bb56273b43a9aa0437c42b9f7e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70b5607f209c4a03b80b63dd3c663682" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:42.179154 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.178999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:57559)
0620 08:09:42.179057 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.179114 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 08:09:42.180109 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.179944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:38463)
0620 08:09:42.180016 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.180055 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 08:09:42.181099 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.180992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.194:38159)
0620 08:09:42.181039 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.181067 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 08:09:42.181236 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.181085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:47073)
0620 08:09:42.181132 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.181206 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 08:09:42.182188 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.182089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.195:34897)
0620 08:09:42.182134 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.182159 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 08:09:42.182672 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.182575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37251 (local address 127.12.132.193:51581)
0620 08:09:42.182608 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.182638 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260620 08:09:42.184067 19730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:42.184163 19730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:42.185448 19730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b66ce99293c4d31949a86e20a77c454: Sending DeleteTablet for 3 replicas of tablet e1f7f997b618456e92862ff117b33261
I20260620 08:09:42.185700 19981 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f7f997b618456e92862ff117b33261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:42 UTC) from {username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:57538
I20260620 08:09:42.185882 19844 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f7f997b618456e92862ff117b33261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:42 UTC) from {username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:50778
I20260620 08:09:42.185907 20304 tablet_replica.cc:330] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: stopping tablet replica
I20260620 08:09:42.185956 20117 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f7f997b618456e92862ff117b33261 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:42 UTC) from {username='oryx', principal='oryx/127.12.132.252@KRBTEST.COM'} at 127.0.0.1:44144
I20260620 08:09:42.185971 20304 raft_consensus.cc:2229] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:42.186025 20304 raft_consensus.cc:2258] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:42.186378 20306 tablet_replica.cc:330] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: stopping tablet replica
I20260620 08:09:42.186466 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19691
I20260620 08:09:42.186739 20304 ts_tablet_manager.cc:1905] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:42.188354 20306 raft_consensus.cc:2229] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:42.188445 20306 raft_consensus.cc:2258] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:42.188768 20306 ts_tablet_manager.cc:1905] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:42.188954 20304 ts_tablet_manager.cc:1918] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:42.189019 20304 log.cc:1196] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/e1f7f997b618456e92862ff117b33261
I20260620 08:09:42.189254 20304 ts_tablet_manager.cc:1939] T e1f7f997b618456e92862ff117b33261 P fa99f38679a04959aa331e13f05064c8: Deleting consensus metadata
I20260620 08:09:42.189841 19717 catalog_manager.cc:4841] TS fa99f38679a04959aa331e13f05064c8 (127.12.132.194:44381): tablet e1f7f997b618456e92862ff117b33261 (table test-table [id=2c454ab7b3f2424e8d83bbc01cb69629]) successfully deleted
I20260620 08:09:42.190418 20306 ts_tablet_manager.cc:1918] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:42.190490 20306 log.cc:1196] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/e1f7f997b618456e92862ff117b33261
I20260620 08:09:42.190728 20306 ts_tablet_manager.cc:1939] T e1f7f997b618456e92862ff117b33261 P 70b5607f209c4a03b80b63dd3c663682: Deleting consensus metadata
I20260620 08:09:42.191252 19718 catalog_manager.cc:4841] TS 70b5607f209c4a03b80b63dd3c663682 (127.12.132.195:33631): tablet e1f7f997b618456e92862ff117b33261 (table test-table [id=2c454ab7b3f2424e8d83bbc01cb69629]) successfully deleted
I20260620 08:09:42.192531 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:42.192398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40273 (local address 127.0.0.1:50786)
0620 08:09:42.192449 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:42.192474 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:40273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 08:09:42.192829 19716 catalog_manager.cc:4584] TS 28e3bb56273b43a9aa0437c42b9f7e23 (127.12.132.193:40273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1f7f997b618456e92862ff117b33261: Network error: Client connection negotiation failed: client connection to 127.12.132.193:40273: connect: Connection refused (error 111)
I20260620 08:09:42.193150 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19918
I20260620 08:09:42.199298 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20054
I20260620 08:09:42.205924 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19412
I20260620 08:09:42.211395 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 19599
2026-06-20T08:09:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6341 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:42 dist-test-slave-z0sp krb5kdc[20309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:42 dist-test-slave-z0sp krb5kdc[20309](info): set up 2 sockets
Jun 20 08:09:42 dist-test-slave-z0sp krb5kdc[20309](info): commencing operation
krb5kdc: starting...
W20260620 08:09:43.266053 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab.
Entry for principal oryx/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab.
I20260620 08:09:43.396071 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39011
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab
--principal=oryx/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:43.476959 20322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:43.477157 20322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:43.477180 20322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:43.478243 20322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:43.478274 20322 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 08:09:43.478288 20322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:43.478302 20322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:43.478313 20322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:43.478325 20322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:43.479744 20322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:39011
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.254.keytab
--principal=oryx/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:43.479941 20322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:43.480099 20322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:43.484764 20327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:43.484886 20322 server_base.cc:1034] running on GCE node
W20260620 08:09:43.484808 20330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:43.484833 20328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:43.485213 20322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:43.485500 20322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:43.486639 20322 hybrid_clock.cc:648] HybridClock initialized: now 1781942983486621 us; error 31 us; skew 500 ppm
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:43.489326 20322 init.cc:377] Logged in from keytab as oryx/127.12.132.254@KRBTEST.COM (short username oryx)
I20260620 08:09:43.489787 20322 webserver.cc:460] Webserver started at http://127.12.132.254:33983/ using document root <none> and password file <none>
I20260620 08:09:43.490022 20322 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:43.490067 20322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:43.490190 20322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:43.491248 20322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "c1880004194543788278d0a12d23c4ad"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "12c14460788d4d981b8ac7565810ae31"
server_key_iv: "58c8cfa74ace4f54e9e59cb8712d3e5c"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.491626 20322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "c1880004194543788278d0a12d23c4ad"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "12c14460788d4d981b8ac7565810ae31"
server_key_iv: "58c8cfa74ace4f54e9e59cb8712d3e5c"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.493038 20322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:43.493777 20337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:43.493939 20322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:43.494041 20322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "c1880004194543788278d0a12d23c4ad"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "12c14460788d4d981b8ac7565810ae31"
server_key_iv: "58c8cfa74ace4f54e9e59cb8712d3e5c"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.494113 20322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:43.504819 20322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:43.506925 20322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:43.507100 20322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:43.511890 20322 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:39011
I20260620 08:09:43.511926 20399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:39011 every 8 connection(s)
I20260620 08:09:43.512259 20322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:43.512914 20400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:43.515021 20400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Bootstrap starting.
I20260620 08:09:43.515539 20400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:43.515765 20400 log.cc:824] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:43.516796 20400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: No bootstrap required, opened a new log
I20260620 08:09:43.519449 20400 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } }
I20260620 08:09:43.519634 20400 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:43.519666 20400 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1880004194543788278d0a12d23c4ad, State: Initialized, Role: FOLLOWER
I20260620 08:09:43.519949 20400 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } }
I20260620 08:09:43.520051 20400 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:43.520078 20400 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:43.520138 20400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:43.520903 20400 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } }
I20260620 08:09:43.521016 20400 leader_election.cc:302] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1880004194543788278d0a12d23c4ad; no voters:
I20260620 08:09:43.521262 20400 leader_election.cc:288] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:43.521410 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20322
I20260620 08:09:43.521488 20403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:43.521585 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:43.521670 20403 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [term 1 LEADER]: Becoming Leader. State: Replica: c1880004194543788278d0a12d23c4ad, State: Running, Role: LEADER
I20260620 08:09:43.521832 12819 external_mini_cluster.cc:1428] Setting key 38eb6e4a52a767b231a0ed7c723a841b
I20260620 08:09:43.521801 20400 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:43.521792 20403 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } }
I20260620 08:09:43.522228 20404 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1880004194543788278d0a12d23c4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } } }
I20260620 08:09:43.522318 20404 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:43.522531 20407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:43.523031 20407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:43.523206 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1880004194543788278d0a12d23c4ad. Latest consensus state: current_term: 1 leader_uuid: "c1880004194543788278d0a12d23c4ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1880004194543788278d0a12d23c4ad" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39011 } } }
I20260620 08:09:43.523344 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:43.524753 20407 catalog_manager.cc:1347] Generated new cluster ID: 34f8fa0d20d6478cbac78d6e7d67d151
I20260620 08:09:43.524817 20407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:43.535436 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.522617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.12.132.254:39011)
0620 08:09:43.523755 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:43.523765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:43.523793 (+ 28us) server_negotiation.cc:407] Connection header received
0620 08:09:43.523918 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:43.523924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:43.523979 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:43.524109 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:43.525074 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.526402 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.527476 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.528712 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.528815 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.531728 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:43.531732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:43.531732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:43.531744 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:43.533752 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.534123 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.534126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.534127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.534230 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.534424 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.534425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.534426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.534728 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:43.534881 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:43.535055 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 08:09:43.535190 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":80,"threads_started":1}
I20260620 08:09:43.541145 20407 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:43.541666 20407 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:43.552307 20407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Generated new TSK 0
I20260620 08:09:43.552515 20407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
I20260620 08:09:43.587606 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39011
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:43.669059 20426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:43.669257 20426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:43.669282 20426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:43.670349 20426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:43.670382 20426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:43.670403 20426 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:43.670419 20426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:43.671885 20426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:43.672096 20426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:43.672258 20426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:43.677084 20431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:43.677093 20434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:43.677140 20426 server_base.cc:1034] running on GCE node
W20260620 08:09:43.677404 20432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:43.677639 20426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:43.677883 20426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:43.679023 20426 hybrid_clock.cc:648] HybridClock initialized: now 1781942983679003 us; error 36 us; skew 500 ppm
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:43.681630 20426 init.cc:377] Logged in from keytab as oryx/127.12.132.193@KRBTEST.COM (short username oryx)
I20260620 08:09:43.682133 20426 webserver.cc:460] Webserver started at http://127.12.132.193:35871/ using document root <none> and password file <none>
I20260620 08:09:43.682363 20426 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:43.682430 20426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:43.682562 20426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:43.683606 20426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "bdd0259d1a2448b49afb23be7602d4a2"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "f6af1dc5d894e1479f5554d3b255bcf7"
server_key_iv: "6ebc8630afb29237d03c8961eb56bbfd"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.683967 20426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "bdd0259d1a2448b49afb23be7602d4a2"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "f6af1dc5d894e1479f5554d3b255bcf7"
server_key_iv: "6ebc8630afb29237d03c8961eb56bbfd"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.685518 20426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:43.686261 20441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:43.686444 20426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:43.686542 20426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "bdd0259d1a2448b49afb23be7602d4a2"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "f6af1dc5d894e1479f5554d3b255bcf7"
server_key_iv: "6ebc8630afb29237d03c8961eb56bbfd"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.686626 20426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:43.718143 20426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:43.720208 20426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:43.720374 20426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:43.721035 20426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:43.721092 20426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:43.721130 20426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:43.721158 20426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:43.728955 20426 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:37439
I20260620 08:09:43.729035 20556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:37439 every 8 connection(s)
I20260620 08:09:43.729355 20426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:43.732013 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.721722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:55257 (local address 127.12.132.254:39011)
0620 08:09:43.721811 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:43.721813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:43.722015 (+ 202us) server_negotiation.cc:407] Connection header received
0620 08:09:43.722540 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:43.722542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:43.722554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:43.722604 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:43.723393 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.723986 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.725047 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.725258 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.725350 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.729131 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:43.729133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:43.729133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:43.729142 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:43.730715 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.731135 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.731136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.731137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.731193 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.731462 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.731463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.731463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.731575 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:43.731631 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:43.731874 (+ 243us) server_negotiation.cc:299] Negotiation successful
0620 08:09:43.731949 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:09:43.732168 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.721650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.193:55257)
0620 08:09:43.721844 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:43.721889 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:43.722426 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:43.722693 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:43.722696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:43.723075 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:43.723293 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.723298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.724075 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.724076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.724949 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.724951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.725362 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.725363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.725530 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.726084 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:43.726105 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:43.729037 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:43.730821 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.730827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.730829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.731049 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.731263 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.731263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.731263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.731360 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.731633 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:43.731637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:43.731745 (+ 108us) client_negotiation.cc:769] Sending connection context
0620 08:09:43.731826 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:09:43.731942 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260620 08:09:43.732698 20557 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39011
I20260620 08:09:43.732782 20557 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:43.732969 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20426
I20260620 08:09:43.732992 20557 heartbeater.cc:511] Master 127.12.132.254:39011 requested a full tablet report, sending...
I20260620 08:09:43.733088 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:43.733325 12819 external_mini_cluster.cc:1428] Setting key dc8537eff2becb6db57f7ef9987f96dd
I20260620 08:09:43.733839 20354 ts_manager.cc:194] Registered new tserver with Master: bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439)
I20260620 08:09:43.734720 20354 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:55257
I20260620 08:09:43.740311 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.733712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.193:54069)
0620 08:09:43.733834 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:43.733843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:43.733910 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:43.734060 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:43.734061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:43.734261 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:43.734319 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.734321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.735012 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.735013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.735782 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.735784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.736174 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.736175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.736240 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.736707 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:43.736711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:43.736974 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:43.738987 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.738989 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.738990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.739260 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.739600 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.739602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.739602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.739690 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.740026 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:43.740027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:43.740069 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 08:09:43.740141 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 08:09:43.740231 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260620 08:09:43.740392 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.733705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:54069 (local address 127.12.132.254:39011)
0620 08:09:43.733741 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:43.733743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:43.733890 (+ 147us) server_negotiation.cc:407] Connection header received
0620 08:09:43.733969 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:43.733969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:43.733978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:43.734004 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:43.734410 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.734936 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.735885 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.736089 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.736173 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.737059 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:43.737061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:43.737061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:43.737068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:43.738842 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.739379 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.739381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.739381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.739447 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.739769 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.739770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.739770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.739916 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:43.739995 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:43.740188 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 08:09:43.740272 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
I20260620 08:09:43.761880 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39011
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:43.843577 20566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:43.843771 20566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:43.843794 20566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:43.844942 20566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:43.844981 20566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:43.845001 20566 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:43.845018 20566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:43.846422 20566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:43.846588 20566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:43.846753 20566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:43.851516 20572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:43.851519 20571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:43.851534 20574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:43.851951 20566 server_base.cc:1034] running on GCE node
I20260620 08:09:43.852159 20566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:43.852393 20566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:43.853549 20566 hybrid_clock.cc:648] HybridClock initialized: now 1781942983853527 us; error 32 us; skew 500 ppm
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:43.855952 20566 init.cc:377] Logged in from keytab as oryx/127.12.132.194@KRBTEST.COM (short username oryx)
I20260620 08:09:43.856350 20566 webserver.cc:460] Webserver started at http://127.12.132.194:42905/ using document root <none> and password file <none>
I20260620 08:09:43.856617 20566 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:43.856683 20566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:43.856822 20566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:43.857818 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "a4439f5ce8044ea1a53dca4619c33e89"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "3bb8936616991c87d330a1dd1b5d5939"
server_key_iv: "2c199eae5fac7d52e8e78d8bdc1aff60"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.858315 20566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "a4439f5ce8044ea1a53dca4619c33e89"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "3bb8936616991c87d330a1dd1b5d5939"
server_key_iv: "2c199eae5fac7d52e8e78d8bdc1aff60"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.859642 20566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:43.860389 20581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:43.860565 20566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:43.860666 20566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "a4439f5ce8044ea1a53dca4619c33e89"
format_stamp: "Formatted at 2026-06-20 08:09:43 on dist-test-slave-z0sp"
server_key: "3bb8936616991c87d330a1dd1b5d5939"
server_key_iv: "2c199eae5fac7d52e8e78d8bdc1aff60"
server_key_version: "encryptionkey@0"
I20260620 08:09:43.860744 20566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:43.881043 20566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:43.883740 20566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:43.883930 20566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:43.884438 20566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:43.884501 20566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:43.884577 20566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:43.884603 20566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:43 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:43.892875 20566 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:43249
I20260620 08:09:43.892997 20696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:43249 every 8 connection(s)
I20260620 08:09:43.893338 20566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:43.895571 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.885219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40373 (local address 127.12.132.254:39011)
0620 08:09:43.885283 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:43.885285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:43.885481 (+ 196us) server_negotiation.cc:407] Connection header received
0620 08:09:43.886142 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:43.886143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:43.886155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:43.886213 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:43.887077 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.888007 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.889231 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.889477 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.889559 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.892467 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:43.892470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:43.892470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:43.892481 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:43.894260 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.894710 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.894711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.894711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.894763 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.894983 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.894983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.894983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.895081 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:43.895131 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:43.895439 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 08:09:43.895506 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:43.895845 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.885181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.194:40373)
0620 08:09:43.885366 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:43.885401 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:43.886046 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:43.886305 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:43.886310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:43.886694 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:43.886965 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.886970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.888104 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.888105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.889115 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.889118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.889579 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.889580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.889705 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.890293 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:43.890312 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:43.892327 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:43.894383 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.894390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.894391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.894629 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.894831 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.894832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.894832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.894926 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.895159 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:43.895165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:43.895286 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 08:09:43.895385 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 08:09:43.895507 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260620 08:09:43.896422 20697 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39011
I20260620 08:09:43.896653 20697 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:43.896871 20697 heartbeater.cc:511] Master 127.12.132.254:39011 requested a full tablet report, sending...
I20260620 08:09:43.897317 20353 ts_manager.cc:194] Registered new tserver with Master: a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249)
I20260620 08:09:43.897866 20353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:40373
I20260620 08:09:43.898834 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20566
I20260620 08:09:43.898985 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:43.899209 12819 external_mini_cluster.cc:1428] Setting key 1192b94c3cb336adf91a8bf731777313
I20260620 08:09:43.903729 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.897673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:58583 (local address 127.12.132.254:39011)
0620 08:09:43.897711 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:43.897713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:43.897784 (+ 71us) server_negotiation.cc:407] Connection header received
0620 08:09:43.897842 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:43.897843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:43.897851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:43.897898 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:43.898305 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.898854 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.899804 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.900088 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.900142 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.901328 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:43.901329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:43.901330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:43.901335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:43.902690 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.903055 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.903056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.903056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.903103 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:43.903298 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:43.903298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:43.903299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:43.903418 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:43.903469 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:43.903623 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 08:09:43.903674 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 08:09:43.903880 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:43.897602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.194:58583)
0620 08:09:43.897729 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:43.897739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:43.897790 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:43.897904 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:43.897905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:43.898149 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:43.898241 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.898244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.899024 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.899025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.899720 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:43.899722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:43.900164 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:43.900165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:43.900247 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:43.900808 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:43.900814 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:43.901214 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:43.902748 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.902749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.902750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.902973 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.903167 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:43.903168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:43.903169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:43.903237 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:43.903482 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:43.903483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:43.903531 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:09:43.903607 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 08:09:43.903722 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
I20260620 08:09:43.928205 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:39011
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:44.007653 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:44.007879 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:44.007913 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:44.009176 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:44.009246 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:44.009281 20706 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 08:09:44.009321 20706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:44.010804 20706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.132.254:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:44.011075 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:44.011304 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:44.016130 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:44.016134 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:44.016129 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:44.016566 20706 server_base.cc:1034] running on GCE node
I20260620 08:09:44.016790 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:44.017028 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:44.018203 20706 hybrid_clock.cc:648] HybridClock initialized: now 1781942984018184 us; error 29 us; skew 500 ppm
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:44.020967 20706 init.cc:377] Logged in from keytab as oryx/127.12.132.195@KRBTEST.COM (short username oryx)
I20260620 08:09:44.021386 20706 webserver.cc:460] Webserver started at http://127.12.132.195:44531/ using document root <none> and password file <none>
I20260620 08:09:44.021600 20706 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:44.021667 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:44.021823 20706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:44.023044 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "9d3f09e862324429b827dd18d1a7782a"
format_stamp: "Formatted at 2026-06-20 08:09:44 on dist-test-slave-z0sp"
server_key: "749039d4588dc63756b106542429038b"
server_key_iv: "b181a68251911f0900e11ea7185181c7"
server_key_version: "encryptionkey@0"
I20260620 08:09:44.023406 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "9d3f09e862324429b827dd18d1a7782a"
format_stamp: "Formatted at 2026-06-20 08:09:44 on dist-test-slave-z0sp"
server_key: "749039d4588dc63756b106542429038b"
server_key_iv: "b181a68251911f0900e11ea7185181c7"
server_key_version: "encryptionkey@0"
I20260620 08:09:44.024760 20706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:09:44.025593 20721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.025858 20706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:44.025961 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "9d3f09e862324429b827dd18d1a7782a"
format_stamp: "Formatted at 2026-06-20 08:09:44 on dist-test-slave-z0sp"
server_key: "749039d4588dc63756b106542429038b"
server_key_iv: "b181a68251911f0900e11ea7185181c7"
server_key_version: "encryptionkey@0"
I20260620 08:09:44.026044 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:44.034907 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:44.037685 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:44.037870 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:44.038358 20706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:44.038425 20706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.038473 20706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:44.038506 20706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:44.045718 20706 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:42663
I20260620 08:09:44.045835 20836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:42663 every 8 connection(s)
I20260620 08:09:44.046108 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:09:44.048945 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.039060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:40273 (local address 127.12.132.254:39011)
0620 08:09:44.039108 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.039110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.039558 (+ 448us) server_negotiation.cc:407] Connection header received
0620 08:09:44.040111 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.040113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.040128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.040214 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.041090 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.041938 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.043525 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.043830 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.043923 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.046255 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:44.046257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:44.046257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:44.046265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:44.047555 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.047946 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.047948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.047948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.048007 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.048262 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.048263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.048264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.048384 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:44.048469 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.048771 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.048859 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:44.049124 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.039078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.195:40273)
0620 08:09:44.039399 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.039435 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.039963 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.040304 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.040322 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.040706 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:44.040989 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.040996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.042047 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.042048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.043401 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.043404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.043932 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.043934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.044069 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.044645 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:44.044653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:44.046178 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:44.047642 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.047648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.047650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.047866 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.048081 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.048082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.048082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.048193 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.048459 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:44.048466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:44.048628 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.048734 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.048891 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":222,"threads_started":1}
I20260620 08:09:44.049623 20837 heartbeater.cc:347] Connected to a master server at 127.12.132.254:39011
I20260620 08:09:44.049806 20837 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:44.050010 20837 heartbeater.cc:511] Master 127.12.132.254:39011 requested a full tablet report, sending...
I20260620 08:09:44.050439 20354 ts_manager.cc:194] Registered new tserver with Master: 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.050908 20354 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:40273
I20260620 08:09:44.053242 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20706
I20260620 08:09:44.053347 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:44.053536 12819 external_mini_cluster.cc:1428] Setting key 5eba13fe72a7ec1d7c9b2c7e0e0329a1
I20260620 08:09:44.054889 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:09:44.057436 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.050765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.12.132.195:52399)
0620 08:09:44.050917 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.050929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.050992 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.051146 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.051147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.051331 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:44.051386 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.051387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.052099 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.052100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.052707 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.052709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.053126 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.053126 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.053174 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.053625 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:44.053632 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:44.054016 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:44.056235 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.056236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.056236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.056486 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.056749 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.056749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.056750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.056824 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.057056 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:44.057057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:44.057104 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.057175 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.057306 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260620 08:09:44.057600 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.050863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:52399 (local address 127.12.132.254:39011)
0620 08:09:44.050904 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.050906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.050990 (+ 84us) server_negotiation.cc:407] Connection header received
0620 08:09:44.051079 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.051080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.051091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.051147 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.051460 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.051991 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.052784 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.053032 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.053124 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.054213 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:44.054215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:44.054216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:44.054227 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:44.056098 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.056598 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.056600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.056600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.056671 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.056898 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.056898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.056898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.056993 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:44.057045 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.057320 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.057455 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 08:09:44.077986 20413 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:44.072241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.12.132.254:39011)
0620 08:09:44.072357 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.072360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.072382 (+ 22us) server_negotiation.cc:407] Connection header received
0620 08:09:44.072452 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.072454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.072467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.072575 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.073187 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.073683 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.074486 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.074665 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.074725 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.077887 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36304: BlockingRecv error: recv got EOF from 127.0.0.1:36304 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:44.085474 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.078751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.12.132.254:39011)
0620 08:09:44.078833 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.078835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.078852 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:09:44.078901 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.078902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.078910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.078975 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.079403 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.079871 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.080613 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.080787 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.080852 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.082980 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:44.082991 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:44.082992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:44.082997 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:44.084444 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.084842 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.084843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.084843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.084897 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.085071 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.085072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.085072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.085171 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:44.085246 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.085354 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.085413 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 08:09:44.087242 20354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:44.088883 20354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:44.095108 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42663 (local address 127.0.0.1:39522)
0620 08:09:44.091249 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.091289 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.091355 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.091702 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.091705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.091707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.091789 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.091791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.092782 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.092783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.093882 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.093884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.094804 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.094805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.094893 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.094909 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.094985 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.095035 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":43,"threads_started":1}
I20260620 08:09:44.095145 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.12.132.195:42663)
0620 08:09:44.091561 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.091564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.091571 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:44.091599 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.091601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.091613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.091679 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.091887 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.092701 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.093979 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.094722 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.094814 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.094865 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.095013 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.095054 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":104,"threads_started":1}
I20260620 08:09:44.095582 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:37439 (local address 127.0.0.1:35242)
0620 08:09:44.091355 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.091365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.091404 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.091972 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.091972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.091974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.092005 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.092007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.093259 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.093260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.094384 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.094387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.095363 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.095364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.095427 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.095431 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.095480 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.095518 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":96,"threads_started":1}
I20260620 08:09:44.095894 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.12.132.193:37439)
0620 08:09:44.091829 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.091833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.091842 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:44.091881 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.091883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.091899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.091974 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.092076 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.093158 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.094476 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.095187 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.095279 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.095329 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.095742 (+ 413us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.095785 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":74,"threads_started":1}
I20260620 08:09:44.095790 20771 tablet_service.cc:1467] Processing CreateTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 (DEFAULT_TABLE table=test-table [id=d87ef151880748e3858f2181b8504ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.096076 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:43249 (local address 127.0.0.1:44334)
0620 08:09:44.091458 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.091466 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.091497 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.091732 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.091733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.091736 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.091788 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.091790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.093261 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.093262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.094557 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.094559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.095645 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.095646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.095796 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.095800 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.095863 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.095903 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":49,"thread_start_us":91,"threads_started":1}
I20260620 08:09:44.096163 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ca7055a4da42f2ba0f97a00a107651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.096309 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.091325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44334 (local address 127.12.132.194:43249)
0620 08:09:44.091489 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.091493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.091502 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:44.091597 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.091600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.091615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.091711 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.091878 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.093175 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.094655 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.095542 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.095740 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.095806 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.095876 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.095913 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260620 08:09:44.096407 20491 tablet_service.cc:1467] Processing CreateTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 (DEFAULT_TABLE table=test-table [id=d87ef151880748e3858f2181b8504ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.096729 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ca7055a4da42f2ba0f97a00a107651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.097002 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 (DEFAULT_TABLE table=test-table [id=d87ef151880748e3858f2181b8504ffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.097321 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ca7055a4da42f2ba0f97a00a107651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.099979 20865 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Bootstrap starting.
I20260620 08:09:44.100477 20866 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Bootstrap starting.
I20260620 08:09:44.100644 20867 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Bootstrap starting.
I20260620 08:09:44.100665 20865 tablet_bootstrap.cc:654] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.100976 20865 log.cc:824] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:44.101181 20866 tablet_bootstrap.cc:654] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.101387 20867 tablet_bootstrap.cc:654] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.101490 20866 log.cc:824] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:44.101688 20867 log.cc:824] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:44.101778 20865 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: No bootstrap required, opened a new log
I20260620 08:09:44.101862 20865 ts_tablet_manager.cc:1397] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:44.102538 20867 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: No bootstrap required, opened a new log
I20260620 08:09:44.102622 20867 ts_tablet_manager.cc:1397] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:09:44.103961 20865 raft_consensus.cc:348] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.104116 20865 raft_consensus.cc:374] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.104151 20865 raft_consensus.cc:729] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3f09e862324429b827dd18d1a7782a, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.104267 20865 consensus_queue.cc:260] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.104532 20865 ts_tablet_manager.cc:1428] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:09:44.104626 20837 heartbeater.cc:503] Master 127.12.132.254:39011 was elected leader, sending a full tablet report...
I20260620 08:09:44.105968 20866 tablet_bootstrap.cc:492] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: No bootstrap required, opened a new log
I20260620 08:09:44.106051 20866 ts_tablet_manager.cc:1397] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260620 08:09:44.106143 20867 raft_consensus.cc:348] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.106383 20867 raft_consensus.cc:374] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.106419 20867 raft_consensus.cc:729] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.106570 20867 consensus_queue.cc:260] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.106945 20867 ts_tablet_manager.cc:1428] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 08:09:44.107105 20557 heartbeater.cc:503] Master 127.12.132.254:39011 was elected leader, sending a full tablet report...
I20260620 08:09:44.108142 20866 raft_consensus.cc:348] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.108299 20866 raft_consensus.cc:374] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.108343 20866 raft_consensus.cc:729] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.108475 20866 consensus_queue.cc:260] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.108723 20866 ts_tablet_manager.cc:1428] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:09:44.108839 20697 heartbeater.cc:503] Master 127.12.132.254:39011 was elected leader, sending a full tablet report...
W20260620 08:09:44.144151 20698 tablet.cc:2369] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:44.167076 20872 raft_consensus.cc:482] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:44.167268 20872 raft_consensus.cc:504] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.167904 20872 leader_election.cc:288] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663), a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249)
I20260620 08:09:44.172410 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.168436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:60009 (local address 127.12.132.194:43249)
0620 08:09:44.168529 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.168532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.168540 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:44.168575 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.168575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.168583 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.168628 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.168832 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.169645 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.171351 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.171948 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.172042 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.172050 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.172294 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.172336 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:09:44.172410 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.168238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:45855 (local address 127.12.132.195:42663)
0620 08:09:44.168314 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.168316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.168327 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:44.168360 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.168361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.168373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.168432 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.168716 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.169629 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.171352 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.171961 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.172040 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.172050 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.172294 (+ 244us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.172336 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 08:09:44.172423 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.168142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42663 (local address 127.12.132.193:45855)
0620 08:09:44.168211 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.168226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.168275 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.168452 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.168453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.168455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.168587 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.168591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.169839 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.169841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.171219 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.171223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.172088 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.172090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.172196 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.172200 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.172272 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.172319 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":70}
I20260620 08:09:44.172434 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.168181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:43249 (local address 127.12.132.193:60009)
0620 08:09:44.168402 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.168411 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.168458 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.168659 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.168660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.168662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.168751 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.168752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.169838 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.169841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.171219 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.171223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.172089 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.172090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.172195 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.172200 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.172272 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.172319 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":89,"thread_start_us":151,"threads_started":1}
I20260620 08:09:44.172987 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ca7055a4da42f2ba0f97a00a107651" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a" is_pre_election: true
I20260620 08:09:44.172982 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ca7055a4da42f2ba0f97a00a107651" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89" is_pre_election: true
I20260620 08:09:44.173214 20651 raft_consensus.cc:2454] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.173214 20791 raft_consensus.cc:2454] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.173552 20444 leader_election.cc:302] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.173784 20872 raft_consensus.cc:2781] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:44.173905 20872 raft_consensus.cc:482] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:44.173940 20872 raft_consensus.cc:3037] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.175297 20872 raft_consensus.cc:504] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.175513 20872 leader_election.cc:288] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Requested vote from peers 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663), a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249)
I20260620 08:09:44.175712 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ca7055a4da42f2ba0f97a00a107651" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89"
I20260620 08:09:44.175853 20651 raft_consensus.cc:3037] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.175796 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ca7055a4da42f2ba0f97a00a107651" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a"
I20260620 08:09:44.175947 20791 raft_consensus.cc:3037] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.176698 20651 raft_consensus.cc:2454] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.176950 20443 leader_election.cc:302] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4439f5ce8044ea1a53dca4619c33e89, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.177119 20872 raft_consensus.cc:2781] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:44.177130 20791 raft_consensus.cc:2454] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.177380 20872 raft_consensus.cc:686] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 LEADER]: Becoming Leader. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Running, Role: LEADER
I20260620 08:09:44.177507 20872 consensus_queue.cc:237] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } }
I20260620 08:09:44.178581 20354 catalog_manager.cc:5495] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:44.191763 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.189433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.12.132.193:37439)
0620 08:09:44.189494 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.189495 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.189503 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:44.189530 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.189530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.189545 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.189602 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.189702 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.190156 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.190918 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.191095 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.191149 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.191341 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.191428 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.191498 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.191630 (+ 132us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.191648 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 08:09:44.193425 20651 raft_consensus.cc:1261] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.193435 20791 raft_consensus.cc:1261] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.193755 20872 consensus_queue.cc:1035] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.193907 20875 consensus_queue.cc:1035] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.196331 20878 mvcc.cc:205] Tried to move back new op lower bound from 7298838463253798912 to 7298838463192518656. Current Snapshot: MvccSnapshot[applied={T|T < 7298838463253798912}]
I20260620 08:09:44.197003 20879 mvcc.cc:205] Tried to move back new op lower bound from 7298838463253798912 to 7298838463192518656. Current Snapshot: MvccSnapshot[applied={T|T < 7298838463253798912 or (T in {7298838463253798912})}]
I20260620 08:09:44.199038 20353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.199147 20353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.199862 20353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Sending DeleteTablet for 3 replicas of tablet 77ca7055a4da42f2ba0f97a00a107651
I20260620 08:09:44.200176 20491 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35242
I20260620 08:09:44.200176 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44334
I20260620 08:09:44.200523 20771 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ca7055a4da42f2ba0f97a00a107651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:39522
I20260620 08:09:44.200760 20889 tablet_replica.cc:330] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: stopping tablet replica
I20260620 08:09:44.200727 20353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 08:09:44.200858 20889 raft_consensus.cc:2229] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 08:09:44.200865 20353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:44.201071 20888 tablet_replica.cc:330] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: stopping tablet replica
I20260620 08:09:44.201150 20888 raft_consensus.cc:2229] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:44.201262 20888 raft_consensus.cc:2258] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.200968 20889 raft_consensus.cc:2258] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.200780 20890 tablet_replica.cc:330] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: stopping tablet replica
I20260620 08:09:44.201540 20890 raft_consensus.cc:2229] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.201905 20890 raft_consensus.cc:2258] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.202345 20889 ts_tablet_manager.cc:1905] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.202375 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 (DEFAULT_TABLE table=test-table [id=022d3ecbd9244432a24d302f36892d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.202455 20771 tablet_service.cc:1467] Processing CreateTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 (DEFAULT_TABLE table=test-table [id=022d3ecbd9244432a24d302f36892d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.202494 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622497915bcd439d8730dbf4bf9c96d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.202556 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622497915bcd439d8730dbf4bf9c96d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.203843 20866 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Bootstrap starting.
I20260620 08:09:44.203997 20865 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Bootstrap starting.
I20260620 08:09:44.203995 20889 ts_tablet_manager.cc:1918] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.204061 20889 log.cc:1196] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/77ca7055a4da42f2ba0f97a00a107651
I20260620 08:09:44.204233 20866 tablet_bootstrap.cc:654] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.204326 20889 ts_tablet_manager.cc:1939] T 77ca7055a4da42f2ba0f97a00a107651 P a4439f5ce8044ea1a53dca4619c33e89: Deleting consensus metadata
I20260620 08:09:44.204437 20865 tablet_bootstrap.cc:654] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.204818 20339 catalog_manager.cc:4841] TS a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249): tablet 77ca7055a4da42f2ba0f97a00a107651 (table test-table [id=d87ef151880748e3858f2181b8504ffe]) successfully deleted
I20260620 08:09:44.204985 20491 tablet_service.cc:1467] Processing CreateTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 (DEFAULT_TABLE table=test-table [id=022d3ecbd9244432a24d302f36892d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.205098 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 622497915bcd439d8730dbf4bf9c96d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.205132 20890 ts_tablet_manager.cc:1905] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.205121 20888 ts_tablet_manager.cc:1905] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.206521 20867 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Bootstrap starting.
I20260620 08:09:44.206523 20890 ts_tablet_manager.cc:1918] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.206586 20890 log.cc:1196] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/77ca7055a4da42f2ba0f97a00a107651
I20260620 08:09:44.206853 20890 ts_tablet_manager.cc:1939] T 77ca7055a4da42f2ba0f97a00a107651 P 9d3f09e862324429b827dd18d1a7782a: Deleting consensus metadata
I20260620 08:09:44.206882 20888 ts_tablet_manager.cc:1918] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.207046 20888 log.cc:1196] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/77ca7055a4da42f2ba0f97a00a107651
I20260620 08:09:44.206982 20867 tablet_bootstrap.cc:654] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.207216 20340 catalog_manager.cc:4841] TS 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663): tablet 77ca7055a4da42f2ba0f97a00a107651 (table test-table [id=d87ef151880748e3858f2181b8504ffe]) successfully deleted
I20260620 08:09:44.207465 20888 ts_tablet_manager.cc:1939] T 77ca7055a4da42f2ba0f97a00a107651 P bdd0259d1a2448b49afb23be7602d4a2: Deleting consensus metadata
I20260620 08:09:44.207849 20867 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: No bootstrap required, opened a new log
I20260620 08:09:44.207898 20867 ts_tablet_manager.cc:1397] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:44.207885 20341 catalog_manager.cc:4841] TS bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439): tablet 77ca7055a4da42f2ba0f97a00a107651 (table test-table [id=d87ef151880748e3858f2181b8504ffe]) successfully deleted
I20260620 08:09:44.208102 20865 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: No bootstrap required, opened a new log
I20260620 08:09:44.208060 20867 raft_consensus.cc:348] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.208139 20867 raft_consensus.cc:374] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.208149 20865 ts_tablet_manager.cc:1397] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260620 08:09:44.208165 20867 raft_consensus.cc:729] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.208212 20867 consensus_queue.cc:260] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.208309 20867 ts_tablet_manager.cc:1428] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.208280 20865 raft_consensus.cc:348] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.208343 20865 raft_consensus.cc:374] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.208369 20865 raft_consensus.cc:729] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3f09e862324429b827dd18d1a7782a, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.208412 20865 consensus_queue.cc:260] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.208540 20865 ts_tablet_manager.cc:1428] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.216540 20866 tablet_bootstrap.cc:492] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: No bootstrap required, opened a new log
I20260620 08:09:44.216602 20866 ts_tablet_manager.cc:1397] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20260620 08:09:44.216753 20866 raft_consensus.cc:348] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.216825 20866 raft_consensus.cc:374] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.216846 20866 raft_consensus.cc:729] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.216886 20866 consensus_queue.cc:260] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.216981 20866 ts_tablet_manager.cc:1428] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260620 08:09:44.230504 20558 tablet.cc:2369] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:44.231038 20873 raft_consensus.cc:482] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:44.231123 20873 raft_consensus.cc:504] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.231457 20873 leader_election.cc:288] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.235451 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.231945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:36335 (local address 127.12.132.195:42663)
0620 08:09:44.231988 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.231990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.232038 (+ 48us) server_negotiation.cc:407] Connection header received
0620 08:09:44.232118 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.232120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.232129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.232182 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.232379 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.233031 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.234481 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.235043 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.235127 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.235136 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.235337 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.235378 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:09:44.235494 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.231620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42663 (local address 127.12.132.194:36335)
0620 08:09:44.231975 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.231987 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.232063 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.232194 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.232194 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.232196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.232285 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.232286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.233110 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.233111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.234399 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.234401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.235161 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.235162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.235228 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.235231 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.235329 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.235374 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260620 08:09:44.236083 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622497915bcd439d8730dbf4bf9c96d8" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a" is_pre_election: true
I20260620 08:09:44.236125 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.231602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:37439 (local address 127.12.132.194:52893)
0620 08:09:44.231672 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.231686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.231899 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.232238 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.232239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.232241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:09:44.232300 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.232301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.233257 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.233258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.234399 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.234401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.235887 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.235888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.235967 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.235971 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.236018 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.236056 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":7}
I20260620 08:09:44.236198 20791 raft_consensus.cc:2454] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 0.
I20260620 08:09:44.236397 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.232028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:52893 (local address 127.12.132.193:37439)
0620 08:09:44.232090 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.232092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.232100 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:44.232125 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.232126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.232135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.232192 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:09:44.232382 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.233180 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.235168 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.235766 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.235833 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.235841 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.236289 (+ 448us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.236326 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 08:09:44.236392 20584 leader_election.cc:302] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, a4439f5ce8044ea1a53dca4619c33e89; no voters:
I20260620 08:09:44.236557 20873 raft_consensus.cc:2781] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:44.236589 20873 raft_consensus.cc:482] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:44.236606 20873 raft_consensus.cc:3037] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.236617 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622497915bcd439d8730dbf4bf9c96d8" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd0259d1a2448b49afb23be7602d4a2" is_pre_election: true
I20260620 08:09:44.236723 20511 raft_consensus.cc:2454] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 0.
I20260620 08:09:44.237371 20873 raft_consensus.cc:504] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.237530 20873 leader_election.cc:288] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 election: Requested vote from peers bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.237651 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622497915bcd439d8730dbf4bf9c96d8" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd0259d1a2448b49afb23be7602d4a2"
I20260620 08:09:44.237689 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "622497915bcd439d8730dbf4bf9c96d8" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a"
I20260620 08:09:44.237725 20511 raft_consensus.cc:3037] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.237756 20791 raft_consensus.cc:3037] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.238359 20791 raft_consensus.cc:2454] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 1.
I20260620 08:09:44.238500 20511 raft_consensus.cc:2454] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 1.
I20260620 08:09:44.238524 20584 leader_election.cc:302] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, a4439f5ce8044ea1a53dca4619c33e89; no voters:
I20260620 08:09:44.238597 20873 raft_consensus.cc:2781] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:44.238725 20873 raft_consensus.cc:686] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 LEADER]: Becoming Leader. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Running, Role: LEADER
I20260620 08:09:44.238797 20873 consensus_queue.cc:237] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.239471 20353 catalog_manager.cc:5495] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "a4439f5ce8044ea1a53dca4619c33e89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } health_report { overall_health: UNKNOWN } } }
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942983, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.254@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:44.272873 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.265200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:39011 (local address 127.0.0.1:36328)
0620 08:09:44.265360 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.265375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.265431 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.265597 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.265598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.265866 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:44.265909 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.265911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.266859 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.266860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.267386 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.267387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.267868 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.267869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.267916 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.268475 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:44.268479 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:44.269933 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:44.271727 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.271733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.271734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.271914 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.272129 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:44.272130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:44.272130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:44.272184 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:44.272455 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:44.272461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:44.272647 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.272715 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.272800 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260620 08:09:44.273229 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.265275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.12.132.254:39011)
0620 08:09:44.265337 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.265339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.265482 (+ 143us) server_negotiation.cc:407] Connection header received
0620 08:09:44.265511 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.265512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.265524 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.265582 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.266003 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.266774 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.267471 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.267766 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.267840 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.270019 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:44.270021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:44.270021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:44.270028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:44.271637 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.271994 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.271995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.271996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.272058 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.272249 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.272250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.272251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.272390 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:44.272461 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.272749 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.273014 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":30}
I20260620 08:09:44.275455 20353 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:36328:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 08:09:44.275673 20353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:44.277338 20491 tablet_service.cc:1467] Processing CreateTablet for tablet ff3269ef5c0a42df92e55b4b55d4d8a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94d315d60f4c4fc88e4b77e48a657719]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:44.277382 20631 tablet_service.cc:1467] Processing CreateTablet for tablet ff3269ef5c0a42df92e55b4b55d4d8a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94d315d60f4c4fc88e4b77e48a657719]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:44.277508 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3269ef5c0a42df92e55b4b55d4d8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.277519 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3269ef5c0a42df92e55b4b55d4d8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.278836 20867 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: Bootstrap starting.
I20260620 08:09:44.279008 20866 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: Bootstrap starting.
I20260620 08:09:44.279094 20771 tablet_service.cc:1467] Processing CreateTablet for tablet ff3269ef5c0a42df92e55b4b55d4d8a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=94d315d60f4c4fc88e4b77e48a657719]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 08:09:44.279227 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff3269ef5c0a42df92e55b4b55d4d8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.279253 20867 tablet_bootstrap.cc:654] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.279405 20866 tablet_bootstrap.cc:654] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.280339 20866 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: No bootstrap required, opened a new log
I20260620 08:09:44.280406 20866 ts_tablet_manager.cc:1397] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:44.280572 20866 raft_consensus.cc:348] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.280650 20866 raft_consensus.cc:374] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.280676 20866 raft_consensus.cc:729] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.280711 20866 consensus_queue.cc:260] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.280804 20866 ts_tablet_manager.cc:1428] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:44.280900 20873 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.280998 20873 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.281368 20865 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: Bootstrap starting.
I20260620 08:09:44.281822 20865 tablet_bootstrap.cc:654] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.282258 20867 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: No bootstrap required, opened a new log
I20260620 08:09:44.282310 20867 ts_tablet_manager.cc:1397] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 08:09:44.282449 20867 raft_consensus.cc:348] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.282524 20867 raft_consensus.cc:374] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.282543 20867 raft_consensus.cc:729] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.282582 20865 tablet_bootstrap.cc:492] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: No bootstrap required, opened a new log
I20260620 08:09:44.282590 20867 consensus_queue.cc:260] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.282622 20865 ts_tablet_manager.cc:1397] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:44.282713 20867 ts_tablet_manager.cc:1428] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.282771 20865 raft_consensus.cc:348] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.282833 20865 raft_consensus.cc:374] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.282882 20865 raft_consensus.cc:729] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3f09e862324429b827dd18d1a7782a, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.282812 20875 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.282927 20865 consensus_queue.cc:260] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.282976 20875 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.283023 20865 ts_tablet_manager.cc:1428] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.283209 20871 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.283295 20871 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 08:09:44.296684 20838 tablet.cc:2369] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:09:44.334408 20875 raft_consensus.cc:482] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:44.334590 20875 raft_consensus.cc:504] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.334841 20875 leader_election.cc:288] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.335194 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3269ef5c0a42df92e55b4b55d4d8a7" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a" is_pre_election: true
I20260620 08:09:44.335317 20791 raft_consensus.cc:2454] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.335299 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3269ef5c0a42df92e55b4b55d4d8a7" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89" is_pre_election: true
I20260620 08:09:44.335464 20651 raft_consensus.cc:2454] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.335534 20444 leader_election.cc:302] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.335675 20875 raft_consensus.cc:2781] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:44.335747 20875 raft_consensus.cc:482] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:44.335772 20875 raft_consensus.cc:3037] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.336975 20875 raft_consensus.cc:504] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.337168 20875 leader_election.cc:288] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Requested vote from peers a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.337294 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3269ef5c0a42df92e55b4b55d4d8a7" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89"
I20260620 08:09:44.337392 20651 raft_consensus.cc:3037] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.337345 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff3269ef5c0a42df92e55b4b55d4d8a7" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a"
I20260620 08:09:44.337432 20791 raft_consensus.cc:3037] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.338161 20651 raft_consensus.cc:2454] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.338161 20791 raft_consensus.cc:2454] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.338372 20444 leader_election.cc:302] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.338471 20875 raft_consensus.cc:2781] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:44.338557 20875 raft_consensus.cc:686] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 LEADER]: Becoming Leader. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Running, Role: LEADER
I20260620 08:09:44.338606 20875 consensus_queue.cc:237] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.338884 20883 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: TxnStatusTablet state changed. Reason: New leader bdd0259d1a2448b49afb23be7602d4a2. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.338984 20883 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:44.339116 20907 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 08:09:44.339226 20907 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 08:09:44.339438 20353 catalog_manager.cc:5495] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:44.356868 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.354361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:37439 (local address 127.0.0.1:35268)
0620 08:09:44.354458 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.354485 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.354544 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.354738 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.354739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.354741 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:44.354806 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.354808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.355428 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.355429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.355989 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.355991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.356370 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.356371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.356448 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.356462 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.356694 (+ 232us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.356703 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.356746 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.356794 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260620 08:09:44.356870 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.354448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.12.132.193:37439)
0620 08:09:44.354513 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.354514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.354524 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:44.354643 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.354645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.354657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.354731 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.354881 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.355347 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.356090 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.356282 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.356365 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.356582 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.356624 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.356681 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.356769 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.356794 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 08:09:44.357858 20651 raft_consensus.cc:1261] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.357870 20791 raft_consensus.cc:1261] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.358062 20875 consensus_queue.cc:1035] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.358175 20875 consensus_queue.cc:1035] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.358613 20873 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: TxnStatusTablet state changed. Reason: New leader bdd0259d1a2448b49afb23be7602d4a2. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.358707 20873 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.358965 20871 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: TxnStatusTablet state changed. Reason: New leader bdd0259d1a2448b49afb23be7602d4a2. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.359057 20871 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.359573 20883 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.359649 20875 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.359731 20875 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:44.359658 20883 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P bdd0259d1a2448b49afb23be7602d4a2: This TxnStatusTablet replica's current role is: LEADER
I20260620 08:09:44.360113 20871 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.360188 20871 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P 9d3f09e862324429b827dd18d1a7782a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.360472 20880 mvcc.cc:205] Tried to move back new op lower bound from 7298838463927975936 to 7298838463851294720. Current Snapshot: MvccSnapshot[applied={T|T < 7298838463927975936}]
I20260620 08:09:44.360965 20873 tablet_replica.cc:439] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } } }
I20260620 08:09:44.361044 20873 tablet_replica.cc:442] T ff3269ef5c0a42df92e55b4b55d4d8a7 P a4439f5ce8044ea1a53dca4619c33e89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 08:09:44.364171 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.361465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.12.132.194:43249)
0620 08:09:44.361505 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.361507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.361514 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:44.361553 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.361554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.361571 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.361636 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.361753 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.362479 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.363312 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.363622 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.363691 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.363850 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.363970 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.364038 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.364081 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.364103 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:09:44.368930 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.366608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40587 (local address 127.12.132.193:37439)
0620 08:09:44.366751 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.366753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.366764 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:44.366841 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.366841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.366850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.366916 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.367033 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.367477 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.368205 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.368398 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.368482 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.368690 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.368726 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.368788 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.368849 (+ 61us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.368866 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 08:09:44.368944 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.366393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:37439 (local address 127.12.132.194:40587)
0620 08:09:44.366566 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.366584 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.366757 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.366914 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.366915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.366916 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:44.366973 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.366974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.367561 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.367562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.368105 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.368106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.368483 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.368484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.368576 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.368590 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.368790 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.368793 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.368844 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.368881 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260620 08:09:44.371215 20791 raft_consensus.cc:1261] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Refusing update from remote peer a4439f5ce8044ea1a53dca4619c33e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.371327 20511 raft_consensus.cc:1261] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Refusing update from remote peer a4439f5ce8044ea1a53dca4619c33e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.371599 20873 consensus_queue.cc:1035] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.371745 20895 consensus_queue.cc:1035] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.384230 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.377391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:32927 (local address 127.12.132.194:43249)
0620 08:09:44.377427 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.377429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.380702 (+ 3273us) server_negotiation.cc:407] Connection header received
0620 08:09:44.380749 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.380750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.380768 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.380858 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.381178 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.382037 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.383458 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.383657 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.383730 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.383990 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.384021 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.384081 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.384159 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.384180 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:09:44.384331 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.377214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:43249 (local address 127.12.132.193:32927)
0620 08:09:44.380600 (+ 3386us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.380612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:44.380676 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:44.380948 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:44.380950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:44.380954 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 08:09:44.381046 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.381049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.382337 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.382338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.383326 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:44.383328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.383764 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.383765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:44.383835 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.383866 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.384101 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.384104 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:09:44.384160 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:09:44.384214 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3347,"thread_start_us":85,"threads_started":1}
I20260620 08:09:44.394402 20353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.394531 20353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.395130 20353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Sending DeleteTablet for 3 replicas of tablet 622497915bcd439d8730dbf4bf9c96d8
I20260620 08:09:44.395380 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44334
I20260620 08:09:44.395479 20889 tablet_replica.cc:330] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: stopping tablet replica
I20260620 08:09:44.395490 20771 tablet_service.cc:1514] Processing DeleteTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:39522
I20260620 08:09:44.395521 20889 raft_consensus.cc:2229] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:44.395607 20890 tablet_replica.cc:330] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: stopping tablet replica
I20260620 08:09:44.395627 20889 raft_consensus.cc:2258] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.395660 20890 raft_consensus.cc:2229] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.395702 20890 raft_consensus.cc:2258] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.395983 20890 ts_tablet_manager.cc:1905] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.396008 20889 ts_tablet_manager.cc:1905] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.397807 20890 ts_tablet_manager.cc:1918] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:44.397857 20890 log.cc:1196] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/622497915bcd439d8730dbf4bf9c96d8
I20260620 08:09:44.398080 20890 ts_tablet_manager.cc:1939] T 622497915bcd439d8730dbf4bf9c96d8 P 9d3f09e862324429b827dd18d1a7782a: Deleting consensus metadata
I20260620 08:09:44.398371 20340 catalog_manager.cc:4841] TS 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663): tablet 622497915bcd439d8730dbf4bf9c96d8 (table test-table [id=022d3ecbd9244432a24d302f36892d71]) successfully deleted
I20260620 08:09:44.401598 20491 tablet_service.cc:1514] Processing DeleteTablet for tablet 622497915bcd439d8730dbf4bf9c96d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35242
I20260620 08:09:44.401717 20888 tablet_replica.cc:330] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: stopping tablet replica
I20260620 08:09:44.401767 20888 raft_consensus.cc:2229] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.401811 20888 raft_consensus.cc:2258] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.402029 20889 ts_tablet_manager.cc:1918] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:44.402084 20889 log.cc:1196] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/622497915bcd439d8730dbf4bf9c96d8
I20260620 08:09:44.402096 20888 ts_tablet_manager.cc:1905] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.402294 20889 ts_tablet_manager.cc:1939] T 622497915bcd439d8730dbf4bf9c96d8 P a4439f5ce8044ea1a53dca4619c33e89: Deleting consensus metadata
I20260620 08:09:44.402581 20339 catalog_manager.cc:4841] TS a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249): tablet 622497915bcd439d8730dbf4bf9c96d8 (table test-table [id=022d3ecbd9244432a24d302f36892d71]) successfully deleted
I20260620 08:09:44.403860 20888 ts_tablet_manager.cc:1918] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:44.403911 20888 log.cc:1196] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/622497915bcd439d8730dbf4bf9c96d8
I20260620 08:09:44.404136 20888 ts_tablet_manager.cc:1939] T 622497915bcd439d8730dbf4bf9c96d8 P bdd0259d1a2448b49afb23be7602d4a2: Deleting consensus metadata
I20260620 08:09:44.404621 20341 catalog_manager.cc:4841] TS bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439): tablet 622497915bcd439d8730dbf4bf9c96d8 (table test-table [id=022d3ecbd9244432a24d302f36892d71]) successfully deleted
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 08:09:44.422444 20413 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:44.416953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.12.132.254:39011)
0620 08:09:44.417013 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.417015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.417135 (+ 120us) server_negotiation.cc:407] Connection header received
0620 08:09:44.417221 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.417222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.417233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.417278 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.417848 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.418350 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.419103 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.419306 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.419378 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.422364 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43960: BlockingRecv error: recv got EOF from 127.0.0.1:43960 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM
I20260620 08:09:44.432964 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.423318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.12.132.254:39011)
0620 08:09:44.423356 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.423358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.423570 (+ 212us) server_negotiation.cc:407] Connection header received
0620 08:09:44.423675 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.423675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.423686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.423716 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:44.424358 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.425133 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.426326 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.426707 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.426770 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.429852 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:44.429854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:44.429854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:44.429862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:44.431607 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.432029 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.432030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.432031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.432088 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:44.432289 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:44.432290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:44.432291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:44.432427 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:44.432525 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.432772 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.432853 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:09:44.434995 20353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:44.435173 20353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:44.436442 20491 tablet_service.cc:1467] Processing CreateTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae (DEFAULT_TABLE table=test-table [id=ee9087282c764818bcdf8064d3ad1bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.436442 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae (DEFAULT_TABLE table=test-table [id=ee9087282c764818bcdf8064d3ad1bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.436602 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82dbe7d2e0844c83813ebcb7b7be9bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.436602 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82dbe7d2e0844c83813ebcb7b7be9bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.437928 20867 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Bootstrap starting.
I20260620 08:09:44.438045 20771 tablet_service.cc:1467] Processing CreateTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae (DEFAULT_TABLE table=test-table [id=ee9087282c764818bcdf8064d3ad1bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.438151 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82dbe7d2e0844c83813ebcb7b7be9bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.438225 20867 tablet_bootstrap.cc:654] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.438436 20866 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Bootstrap starting.
I20260620 08:09:44.438833 20866 tablet_bootstrap.cc:654] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.439571 20865 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Bootstrap starting.
I20260620 08:09:44.440075 20867 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: No bootstrap required, opened a new log
I20260620 08:09:44.440119 20867 ts_tablet_manager.cc:1397] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:09:44.440243 20865 tablet_bootstrap.cc:654] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.440230 20867 raft_consensus.cc:348] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.440302 20867 raft_consensus.cc:374] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.440328 20867 raft_consensus.cc:729] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.440382 20867 consensus_queue.cc:260] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.440480 20867 ts_tablet_manager.cc:1428] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 08:09:44.441206 20866 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: No bootstrap required, opened a new log
I20260620 08:09:44.441257 20866 ts_tablet_manager.cc:1397] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 08:09:44.441360 20865 tablet_bootstrap.cc:492] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: No bootstrap required, opened a new log
I20260620 08:09:44.441395 20865 ts_tablet_manager.cc:1397] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:44.441397 20866 raft_consensus.cc:348] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.441469 20866 raft_consensus.cc:374] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.441493 20866 raft_consensus.cc:729] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.441546 20866 consensus_queue.cc:260] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.441540 20865 raft_consensus.cc:348] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.441605 20865 raft_consensus.cc:374] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.441627 20865 raft_consensus.cc:729] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3f09e862324429b827dd18d1a7782a, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.441632 20866 ts_tablet_manager.cc:1428] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.441669 20865 consensus_queue.cc:260] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.441776 20865 ts_tablet_manager.cc:1428] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.678419 20872 raft_consensus.cc:482] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:44.678511 20872 raft_consensus.cc:504] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.678673 20872 leader_election.cc:288] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.678954 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82dbe7d2e0844c83813ebcb7b7be9bae" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89" is_pre_election: true
I20260620 08:09:44.679080 20651 raft_consensus.cc:2454] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.679159 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82dbe7d2e0844c83813ebcb7b7be9bae" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a" is_pre_election: true
I20260620 08:09:44.679265 20443 leader_election.cc:302] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4439f5ce8044ea1a53dca4619c33e89, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.679286 20791 raft_consensus.cc:2454] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 0.
I20260620 08:09:44.679375 20872 raft_consensus.cc:2781] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:44.679428 20872 raft_consensus.cc:482] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:44.679450 20872 raft_consensus.cc:3037] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.680140 20872 raft_consensus.cc:504] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.680276 20872 leader_election.cc:288] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Requested vote from peers a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.680435 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82dbe7d2e0844c83813ebcb7b7be9bae" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4439f5ce8044ea1a53dca4619c33e89"
I20260620 08:09:44.680480 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82dbe7d2e0844c83813ebcb7b7be9bae" candidate_uuid: "bdd0259d1a2448b49afb23be7602d4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a"
I20260620 08:09:44.680668 20651 raft_consensus.cc:3037] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.680711 20791 raft_consensus.cc:3037] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.681699 20651 raft_consensus.cc:2454] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.681699 20791 raft_consensus.cc:2454] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdd0259d1a2448b49afb23be7602d4a2 in term 1.
I20260620 08:09:44.682005 20443 leader_election.cc:302] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4439f5ce8044ea1a53dca4619c33e89, bdd0259d1a2448b49afb23be7602d4a2; no voters:
I20260620 08:09:44.682114 20872 raft_consensus.cc:2781] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:44.682209 20872 raft_consensus.cc:686] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 1 LEADER]: Becoming Leader. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Running, Role: LEADER
I20260620 08:09:44.682263 20872 consensus_queue.cc:237] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.682834 20354 catalog_manager.cc:5495] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "bdd0259d1a2448b49afb23be7602d4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:44.727008 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.724275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.12.132.193:37439)
0620 08:09:44.724352 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.724355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.724374 (+ 19us) server_negotiation.cc:407] Connection header received
0620 08:09:44.724474 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.724475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.724523 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.724593 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.724787 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.725311 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.726202 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.726462 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.726561 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.726752 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.726791 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.726858 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.726910 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.726933 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 08:09:44.727849 20791 raft_consensus.cc:1261] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.727866 20651 raft_consensus.cc:1261] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Refusing update from remote peer bdd0259d1a2448b49afb23be7602d4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.728102 20875 consensus_queue.cc:1035] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.728204 20872 consensus_queue.cc:1035] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.731448 20353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.731557 20353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.732126 20353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Sending DeleteTablet for 3 replicas of tablet 82dbe7d2e0844c83813ebcb7b7be9bae
I20260620 08:09:44.732401 20491 tablet_service.cc:1514] Processing DeleteTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35242
I20260620 08:09:44.732435 20771 tablet_service.cc:1514] Processing DeleteTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:39522
I20260620 08:09:44.732522 20888 tablet_replica.cc:330] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: stopping tablet replica
I20260620 08:09:44.732555 20890 tablet_replica.cc:330] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: stopping tablet replica
I20260620 08:09:44.732576 20888 raft_consensus.cc:2229] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:44.732599 20890 raft_consensus.cc:2229] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.732635 20890 pending_rounds.cc:62] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Trying to abort 2 pending ops.
I20260620 08:09:44.732633 20888 raft_consensus.cc:2258] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.732664 20890 pending_rounds.cc:68] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298838465443454976 op_type: WRITE_OP write_request { tablet_id: "82dbe7d2e0844c83813ebcb7b7be9bae" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\364\227\331\321\006\"8\n\ntest-admin\022*\n ee9087282c764818bcdf8064d3ad1bec\020\001\030\001 \001(\001" signature: "<redacted>""\\\236\010u\261\212\376\326\312a\372\245\033\317-\373\004c\213>\r\022\311\365\035\274\005\005x\023.5\236\337\361\rR\026y\254\323\251c\017G\212\265\245)\025\214\3519\320\375\347\367\221=/V\372\3120" signing_key_seq_num: 0 } } request_id { client_id: "d1d9ba88a34d4688bbd11d43337152f1" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260620 08:09:44.733023 20890 pending_rounds.cc:68] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298838465258921984 op_type: NO_OP noop_request { }
I20260620 08:09:44.733099 20890 raft_consensus.cc:2866] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260620 08:09:44.733130 20890 raft_consensus.cc:2258] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.733146 20888 ts_tablet_manager.cc:1905] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.733774 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 82dbe7d2e0844c83813ebcb7b7be9bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44334
I20260620 08:09:44.733922 20889 tablet_replica.cc:330] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: stopping tablet replica
I20260620 08:09:44.733999 20889 raft_consensus.cc:2229] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.734052 20889 raft_consensus.cc:2258] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.734102 20353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:09:44.734275 20353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:09:44.734335 20889 ts_tablet_manager.cc:1905] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.734704 20890 ts_tablet_manager.cc:1905] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.734768 20888 ts_tablet_manager.cc:1918] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.734818 20888 log.cc:1196] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/82dbe7d2e0844c83813ebcb7b7be9bae
I20260620 08:09:44.735085 20888 ts_tablet_manager.cc:1939] T 82dbe7d2e0844c83813ebcb7b7be9bae P bdd0259d1a2448b49afb23be7602d4a2: Deleting consensus metadata
I20260620 08:09:44.735867 20771 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 (DEFAULT_TABLE table=test-table [id=ce1fdfc5cad34db2a6a19a67f4b6c95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.735993 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb8c7711d843acaf953f0492cb5c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.736322 20889 ts_tablet_manager.cc:1918] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.736397 20889 log.cc:1196] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/82dbe7d2e0844c83813ebcb7b7be9bae
I20260620 08:09:44.736439 20341 catalog_manager.cc:4841] TS bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439): tablet 82dbe7d2e0844c83813ebcb7b7be9bae (table test-table [id=ee9087282c764818bcdf8064d3ad1bec]) successfully deleted
I20260620 08:09:44.736665 20889 ts_tablet_manager.cc:1939] T 82dbe7d2e0844c83813ebcb7b7be9bae P a4439f5ce8044ea1a53dca4619c33e89: Deleting consensus metadata
I20260620 08:09:44.736675 20491 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 (DEFAULT_TABLE table=test-table [id=ce1fdfc5cad34db2a6a19a67f4b6c95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.736768 20491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb8c7711d843acaf953f0492cb5c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.737048 20890 ts_tablet_manager.cc:1918] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:09:44.737100 20890 log.cc:1196] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/82dbe7d2e0844c83813ebcb7b7be9bae
I20260620 08:09:44.737300 20890 ts_tablet_manager.cc:1939] T 82dbe7d2e0844c83813ebcb7b7be9bae P 9d3f09e862324429b827dd18d1a7782a: Deleting consensus metadata
I20260620 08:09:44.737630 20865 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Bootstrap starting.
I20260620 08:09:44.737860 20867 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: Bootstrap starting.
I20260620 08:09:44.737875 20339 catalog_manager.cc:4841] TS a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249): tablet 82dbe7d2e0844c83813ebcb7b7be9bae (table test-table [id=ee9087282c764818bcdf8064d3ad1bec]) successfully deleted
I20260620 08:09:44.737962 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 (DEFAULT_TABLE table=test-table [id=ce1fdfc5cad34db2a6a19a67f4b6c95c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:09:44.738013 20865 tablet_bootstrap.cc:654] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.738058 20631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70eb8c7711d843acaf953f0492cb5c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:44.738144 20867 tablet_bootstrap.cc:654] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.738655 20865 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: No bootstrap required, opened a new log
I20260620 08:09:44.738721 20865 ts_tablet_manager.cc:1397] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:44.738857 20865 raft_consensus.cc:348] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.738938 20865 raft_consensus.cc:374] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.738969 20865 raft_consensus.cc:729] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3f09e862324429b827dd18d1a7782a, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.739022 20865 consensus_queue.cc:260] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.739082 20866 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Bootstrap starting.
I20260620 08:09:44.739116 20865 ts_tablet_manager.cc:1428] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.739399 20866 tablet_bootstrap.cc:654] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:44.739547 20340 catalog_manager.cc:4841] TS 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663): tablet 82dbe7d2e0844c83813ebcb7b7be9bae (table test-table [id=ee9087282c764818bcdf8064d3ad1bec]) successfully deleted
I20260620 08:09:44.739889 20867 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: No bootstrap required, opened a new log
I20260620 08:09:44.739931 20867 ts_tablet_manager.cc:1397] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:09:44.740043 20867 raft_consensus.cc:348] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.740106 20867 raft_consensus.cc:374] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.740150 20867 raft_consensus.cc:729] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd0259d1a2448b49afb23be7602d4a2, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.740198 20867 consensus_queue.cc:260] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.740303 20867 ts_tablet_manager.cc:1428] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.740583 20866 tablet_bootstrap.cc:492] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: No bootstrap required, opened a new log
I20260620 08:09:44.740628 20866 ts_tablet_manager.cc:1397] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260620 08:09:44.740712 20866 raft_consensus.cc:348] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.740763 20866 raft_consensus.cc:374] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:44.740777 20866 raft_consensus.cc:729] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Initialized, Role: FOLLOWER
I20260620 08:09:44.740829 20866 consensus_queue.cc:260] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.740926 20866 ts_tablet_manager.cc:1428] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:44.783742 20916 raft_consensus.cc:482] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:09:44.783849 20916 raft_consensus.cc:504] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.784008 20916 leader_election.cc:288] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.784293 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb8c7711d843acaf953f0492cb5c71" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd0259d1a2448b49afb23be7602d4a2" is_pre_election: true
I20260620 08:09:44.784293 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb8c7711d843acaf953f0492cb5c71" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a" is_pre_election: true
I20260620 08:09:44.784461 20511 raft_consensus.cc:2454] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 0.
I20260620 08:09:44.784461 20791 raft_consensus.cc:2454] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 0.
I20260620 08:09:44.784716 20584 leader_election.cc:302] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, a4439f5ce8044ea1a53dca4619c33e89; no voters:
I20260620 08:09:44.784910 20916 raft_consensus.cc:2781] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:09:44.785002 20916 raft_consensus.cc:482] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:09:44.785044 20916 raft_consensus.cc:3037] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.785784 20916 raft_consensus.cc:504] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.785941 20916 leader_election.cc:288] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 election: Requested vote from peers bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439), 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663)
I20260620 08:09:44.786127 20791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb8c7711d843acaf953f0492cb5c71" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d3f09e862324429b827dd18d1a7782a"
I20260620 08:09:44.786162 20511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70eb8c7711d843acaf953f0492cb5c71" candidate_uuid: "a4439f5ce8044ea1a53dca4619c33e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd0259d1a2448b49afb23be7602d4a2"
I20260620 08:09:44.786226 20791 raft_consensus.cc:3037] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.786254 20511 raft_consensus.cc:3037] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:44.786950 20511 raft_consensus.cc:2454] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 1.
I20260620 08:09:44.786950 20791 raft_consensus.cc:2454] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4439f5ce8044ea1a53dca4619c33e89 in term 1.
I20260620 08:09:44.787134 20584 leader_election.cc:302] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3f09e862324429b827dd18d1a7782a, a4439f5ce8044ea1a53dca4619c33e89; no voters:
I20260620 08:09:44.787271 20916 raft_consensus.cc:2781] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:44.787314 20916 raft_consensus.cc:686] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 LEADER]: Becoming Leader. State: Replica: a4439f5ce8044ea1a53dca4619c33e89, State: Running, Role: LEADER
I20260620 08:09:44.787364 20916 consensus_queue.cc:237] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } }
I20260620 08:09:44.788040 20353 catalog_manager.cc:5495] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "a4439f5ce8044ea1a53dca4619c33e89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4439f5ce8044ea1a53dca4619c33e89" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 43249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 } health_report { overall_health: UNKNOWN } } }
I20260620 08:09:44.805917 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.803397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.12.132.194:43249)
0620 08:09:44.803492 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:44.803494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:44.803501 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:09:44.803518 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:44.803519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:44.803528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:44.803594 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:09:44.803702 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.804187 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.804947 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:44.805154 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:44.805233 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:44.805353 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:09:44.805385 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:09:44.805645 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:44.805746 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 08:09:44.805780 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 08:09:44.808992 20511 raft_consensus.cc:1261] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Refusing update from remote peer a4439f5ce8044ea1a53dca4619c33e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.809170 20791 raft_consensus.cc:1261] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Refusing update from remote peer a4439f5ce8044ea1a53dca4619c33e89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:09:44.809396 20916 consensus_queue.cc:1035] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdd0259d1a2448b49afb23be7602d4a2" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 37439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.809523 20915 consensus_queue.cc:1035] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d3f09e862324429b827dd18d1a7782a" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:09:44.819944 20354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.820043 20354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:09:44.821058 20354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1880004194543788278d0a12d23c4ad: Sending DeleteTablet for 3 replicas of tablet 70eb8c7711d843acaf953f0492cb5c71
I20260620 08:09:44.821297 20491 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35242
I20260620 08:09:44.821344 20771 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:39522
I20260620 08:09:44.821398 20888 tablet_replica.cc:330] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2: stopping tablet replica
I20260620 08:09:44.821446 20890 tablet_replica.cc:330] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: stopping tablet replica
I20260620 08:09:44.821465 20888 raft_consensus.cc:2229] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.821501 20888 raft_consensus.cc:2258] T 70eb8c7711d843acaf953f0492cb5c71 P bdd0259d1a2448b49afb23be7602d4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.821503 20890 raft_consensus.cc:2229] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:09:44.821537 20890 raft_consensus.cc:2258] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.821573 20631 tablet_service.cc:1514] Processing DeleteTablet for tablet 70eb8c7711d843acaf953f0492cb5c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:09:44 UTC) from {username='oryx', principal='oryx/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44334
I20260620 08:09:44.821695 20889 tablet_replica.cc:330] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: stopping tablet replica
I20260620 08:09:44.821760 20889 raft_consensus.cc:2229] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:09:44.821844 20889 raft_consensus.cc:2258] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:09:44.821851 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20426
I20260620 08:09:44.822714 20889 ts_tablet_manager.cc:1905] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:09:44.824560 20889 ts_tablet_manager.cc:1918] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:44.824625 20889 log.cc:1196] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/70eb8c7711d843acaf953f0492cb5c71
I20260620 08:09:44.824852 20889 ts_tablet_manager.cc:1939] T 70eb8c7711d843acaf953f0492cb5c71 P a4439f5ce8044ea1a53dca4619c33e89: Deleting consensus metadata
I20260620 08:09:44.825603 20339 catalog_manager.cc:4841] TS a4439f5ce8044ea1a53dca4619c33e89 (127.12.132.194:43249): tablet 70eb8c7711d843acaf953f0492cb5c71 (table test-table [id=ce1fdfc5cad34db2a6a19a67f4b6c95c]) successfully deleted
I20260620 08:09:44.828759 20890 ts_tablet_manager.cc:1905] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 08:09:44.830073 20341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:37439 (error 108)
I20260620 08:09:44.830202 20890 ts_tablet_manager.cc:1918] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 08:09:44.830267 20890 log.cc:1196] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/70eb8c7711d843acaf953f0492cb5c71
I20260620 08:09:44.830338 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20566
I20260620 08:09:44.830508 20890 ts_tablet_manager.cc:1939] T 70eb8c7711d843acaf953f0492cb5c71 P 9d3f09e862324429b827dd18d1a7782a: Deleting consensus metadata
I20260620 08:09:44.831171 20340 catalog_manager.cc:4841] TS 9d3f09e862324429b827dd18d1a7782a (127.12.132.195:42663): tablet 70eb8c7711d843acaf953f0492cb5c71 (table test-table [id=ce1fdfc5cad34db2a6a19a67f4b6c95c]) successfully deleted
I20260620 08:09:44.832046 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:44.831853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:37439 (local address 127.0.0.1:56292)
0620 08:09:44.831959 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:44.832000 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:37439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":37,"threads_started":1}
W20260620 08:09:44.832188 20341 catalog_manager.cc:4584] TS bdd0259d1a2448b49afb23be7602d4a2 (127.12.132.193:37439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70eb8c7711d843acaf953f0492cb5c71: Network error: Client connection negotiation failed: client connection to 127.12.132.193:37439: connect: Connection refused (error 111)
I20260620 08:09:44.836143 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20706
I20260620 08:09:44.840946 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20322
2026-06-20T08:09:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2627 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20950](info): set up 2 sockets
Jun 20 08:09:44 dist-test-slave-z0sp krb5kdc[20950](info): commencing operation
krb5kdc: starting...
W20260620 08:09:45.905905 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:45 dist-test-slave-z0sp krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:46.040828 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45213
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:45213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:46.129396 20963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:46.129653 20963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:46.129685 20963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:46.131349 20963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:46.131402 20963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:46.131418 20963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:46.131433 20963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:46.131445 20963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:46.133175 20963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:45213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:46.133387 20963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:46.133566 20963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:46.138772 20968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:46.138772 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:46.138764 20969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:46.140069 20963 server_base.cc:1034] running on GCE node
I20260620 08:09:46.140352 20963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:46.140681 20963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:46.141851 20963 hybrid_clock.cc:648] HybridClock initialized: now 1781942986141839 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T08:09:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1307 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:09:46 dist-test-slave-z0sp krb5kdc[20976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:09:46 dist-test-slave-z0sp krb5kdc[20976](info): set up 2 sockets
Jun 20 08:09:46 dist-test-slave-z0sp krb5kdc[20976](info): commencing operation
krb5kdc: starting...
W20260620 08:09:47.193706 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:09:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:09:47.320654 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38953
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:47.402835 20989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:47.403071 20989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:47.403095 20989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:47.404227 20989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:47.404273 20989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:47.404289 20989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:47.404301 20989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:47.404313 20989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:47.405741 20989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:47.405995 20989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:47.406222 20989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:47.411072 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.411160 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.411072 20994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:47.411448 20989 server_base.cc:1034] running on GCE node
I20260620 08:09:47.411743 20989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:47.412164 20989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:47.413450 20989 hybrid_clock.cc:648] HybridClock initialized: now 1781942987413417 us; error 51 us; skew 500 ppm
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:47.418776 20989 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:47.419186 20989 webserver.cc:460] Webserver started at http://127.12.132.254:42735/ using document root <none> and password file <none>
I20260620 08:09:47.419365 20989 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:47.419406 20989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:47.419495 20989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:47.420572 20989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ef9a22e8035d431b96dd656643fe0d57"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "ea307bcdff14cba2c0e75e0ae928ca58"
server_key_iv: "3646e8289cee565ac7a768d2d81628fd"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.420894 20989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ef9a22e8035d431b96dd656643fe0d57"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "ea307bcdff14cba2c0e75e0ae928ca58"
server_key_iv: "3646e8289cee565ac7a768d2d81628fd"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.422422 20989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:47.423444 21004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.423727 20989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:47.423848 20989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ef9a22e8035d431b96dd656643fe0d57"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "ea307bcdff14cba2c0e75e0ae928ca58"
server_key_iv: "3646e8289cee565ac7a768d2d81628fd"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.423926 20989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:47.443459 20989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:47.446383 20989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:47.446599 20989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:47.451520 20989 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38953
I20260620 08:09:47.451609 21056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38953 every 8 connection(s)
I20260620 08:09:47.451957 20989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:47.452915 21057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:09:47.455096 21057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Bootstrap starting.
I20260620 08:09:47.455622 21057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Neither blocks nor log segments found. Creating new log.
I20260620 08:09:47.455896 21057 log.cc:824] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:47.456081 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 20989
I20260620 08:09:47.456305 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:09:47.456640 12819 external_mini_cluster.cc:1428] Setting key c01a51e7d53ee188eacd7420c302e072
I20260620 08:09:47.456748 21057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: No bootstrap required, opened a new log
I20260620 08:09:47.458628 21057 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:47.458762 21057 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:09:47.458786 21057 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9a22e8035d431b96dd656643fe0d57, State: Initialized, Role: FOLLOWER
I20260620 08:09:47.458909 21057 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:47.458977 21057 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:47.459020 21057 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:47.459072 21057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:09:47.459898 21057 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:47.459982 21057 leader_election.cc:302] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9a22e8035d431b96dd656643fe0d57; no voters:
I20260620 08:09:47.460166 21057 leader_election.cc:288] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:09:47.460196 21062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:09:47.460307 21062 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 LEADER]: Becoming Leader. State: Replica: ef9a22e8035d431b96dd656643fe0d57, State: Running, Role: LEADER
I20260620 08:09:47.460415 21057 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:47.460395 21062 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:47.460747 21062 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef9a22e8035d431b96dd656643fe0d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } } }
I20260620 08:09:47.460822 21062 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:47.460944 21062 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9a22e8035d431b96dd656643fe0d57. Latest consensus state: current_term: 1 leader_uuid: "ef9a22e8035d431b96dd656643fe0d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } } }
I20260620 08:09:47.461026 21062 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:47.461308 21068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:47.461787 21068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:47.463691 21068 catalog_manager.cc:1347] Generated new cluster ID: 213444feb8074a419e59469a33449bde
I20260620 08:09:47.463743 21068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:47.469259 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.457268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46260 (local address 127.12.132.254:38953)
0620 08:09:47.457453 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:47.457460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:47.457485 (+ 25us) server_negotiation.cc:407] Connection header received
0620 08:09:47.458623 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:47.458631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:47.458692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:47.458885 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:47.459633 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.460961 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.462132 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.462678 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.463112 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.465994 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:47.465997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:47.465998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:47.466009 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:47.467680 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.468069 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.468072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.468072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.468176 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.468381 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.468382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.468382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.468670 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:47.468840 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:47.468909 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 08:09:47.469044 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260620 08:09:47.474148 21068 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:09:47.474766 21068 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:47.477308 21068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Generated new TSK 0
I20260620 08:09:47.477496 21068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:09:47.507144 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:47.593808 21083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:47.594017 21083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:47.594041 21083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:47.595213 21083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:47.595250 21083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:47.595278 21083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:47.596802 21083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:47.596983 21083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:47.597153 21083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:47.602474 21088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.602474 21091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.602509 21089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:47.602728 21083 server_base.cc:1034] running on GCE node
I20260620 08:09:47.603016 21083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:47.603359 21083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:47.604599 21083 hybrid_clock.cc:648] HybridClock initialized: now 1781942987604577 us; error 65 us; skew 500 ppm
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:47.607215 21083 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:09:47.607741 21083 webserver.cc:460] Webserver started at http://127.12.132.193:43995/ using document root <none> and password file <none>
I20260620 08:09:47.607967 21083 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:47.608036 21083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:47.608188 21083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:47.609313 21083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "2ec218f5d2eb4843a7df3aef0f76c5a7"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "cc231435123e7d1328c0cfa37a8ec130"
server_key_iv: "b6ba9ce9f9e6c2fb5adac9265bf81720"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.609675 21083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "2ec218f5d2eb4843a7df3aef0f76c5a7"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "cc231435123e7d1328c0cfa37a8ec130"
server_key_iv: "b6ba9ce9f9e6c2fb5adac9265bf81720"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.611079 21083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:09:47.611845 21098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.612027 21083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:47.612107 21083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "2ec218f5d2eb4843a7df3aef0f76c5a7"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "cc231435123e7d1328c0cfa37a8ec130"
server_key_iv: "b6ba9ce9f9e6c2fb5adac9265bf81720"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.612159 21083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:47.623728 21083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:47.626017 21083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:47.626130 21083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:47.626348 21083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:47.626626 21083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:47.626659 21083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.626752 21083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:47.626806 21083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.632699 21083 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:41593
I20260620 08:09:47.632823 21211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:41593 every 8 connection(s)
I20260620 08:09:47.633106 21083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:47.642378 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21083
I20260620 08:09:47.642513 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:09:47.642737 12819 external_mini_cluster.cc:1428] Setting key e6093e1f3814573902eae58950a4eb1a
I20260620 08:09:47.642712 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.633885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:55615 (local address 127.12.132.254:38953)
0620 08:09:47.633957 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:47.633959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:47.634143 (+ 184us) server_negotiation.cc:407] Connection header received
0620 08:09:47.634656 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:47.634658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:47.634668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:47.634725 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:47.635635 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.636142 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.637169 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.637384 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.637450 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.639511 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:47.639512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:47.639513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:47.639519 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:47.641052 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.641465 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.641466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.641466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.641523 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.642004 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.642005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.642005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.642114 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:47.642178 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:47.642533 (+ 355us) server_negotiation.cc:299] Negotiation successful
0620 08:09:47.642618 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 08:09:47.643090 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.633835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:55615)
0620 08:09:47.634022 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:47.634058 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:47.634549 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:47.634796 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:47.634799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:47.635350 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:47.635549 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.635553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.636226 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.636227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.637089 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.637092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.637484 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.637485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.637590 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.638067 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:47.638074 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:47.639412 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:47.641145 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.641150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.641152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.641350 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.641662 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.641664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.641664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.641885 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.642235 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:47.642241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:47.642359 (+ 118us) client_negotiation.cc:769] Sending connection context
0620 08:09:47.642516 (+ 157us) client_negotiation.cc:240] Negotiation successful
0620 08:09:47.642678 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260620 08:09:47.643537 21212 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38953
I20260620 08:09:47.643625 21212 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:47.643813 21212 heartbeater.cc:511] Master 127.12.132.254:38953 requested a full tablet report, sending...
I20260620 08:09:47.644292 21021 ts_manager.cc:194] Registered new tserver with Master: 2ec218f5d2eb4843a7df3aef0f76c5a7 (127.12.132.193:41593)
I20260620 08:09:47.644924 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:55615
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:09:47.668329 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:47.749756 21217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:47.749953 21217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:47.749975 21217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:47.751123 21217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:47.751164 21217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:47.751192 21217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:47.752694 21217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:47.752923 21217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:47.753135 21217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:47.758147 21225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.758225 21223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.758225 21222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:47.759169 21217 server_base.cc:1034] running on GCE node
I20260620 08:09:47.759370 21217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:47.759557 21217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:47.760717 21217 hybrid_clock.cc:648] HybridClock initialized: now 1781942987760684 us; error 53 us; skew 500 ppm
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:47.763330 21217 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:09:47.763769 21217 webserver.cc:460] Webserver started at http://127.12.132.194:39829/ using document root <none> and password file <none>
I20260620 08:09:47.764010 21217 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:47.764081 21217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:47.764232 21217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:47.765305 21217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "3a01918899e84e9081351e6a097510e8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "e88827eaba956510a3e4a2af5feac525"
server_key_iv: "26f25773f9bbcbdd192426a532b54bc3"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.765669 21217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "3a01918899e84e9081351e6a097510e8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "e88827eaba956510a3e4a2af5feac525"
server_key_iv: "26f25773f9bbcbdd192426a532b54bc3"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.766988 21217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:47.767858 21232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.768216 21217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:47.768335 21217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "3a01918899e84e9081351e6a097510e8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "e88827eaba956510a3e4a2af5feac525"
server_key_iv: "26f25773f9bbcbdd192426a532b54bc3"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.768411 21217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:47.790264 21217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:47.792637 21217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:47.792807 21217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:47.793041 21217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:47.793354 21217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:47.793413 21217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.793448 21217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:47.793478 21217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.799180 21217 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:43339
I20260620 08:09:47.799223 21345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:43339 every 8 connection(s)
I20260620 08:09:47.799554 21217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:47.803107 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21217
I20260620 08:09:47.803246 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:09:47.803443 12819 external_mini_cluster.cc:1428] Setting key c2a20dc090bf4f3a89ce888575c0ef0f
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:47.811373 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.800190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:51007)
0620 08:09:47.800441 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:47.800551 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:47.801223 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:47.801530 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:47.801534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:47.802217 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:47.802443 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.802448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.803383 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.803384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.804687 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.804691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.805090 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.805091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.805225 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.805782 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:47.805792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:47.807553 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:47.809805 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.809810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.809813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.810055 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.810307 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.810308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.810308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.810424 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.810742 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:47.810746 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:47.810870 (+ 124us) client_negotiation.cc:769] Sending connection context
0620 08:09:47.811058 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 08:09:47.811204 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":109,"threads_started":1}
I20260620 08:09:47.811367 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.800315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:51007 (local address 127.12.132.254:38953)
0620 08:09:47.800432 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:47.800436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:47.800679 (+ 243us) server_negotiation.cc:407] Connection header received
0620 08:09:47.801346 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:47.801347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:47.801361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:47.801426 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:47.802619 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.803260 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.804782 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.804994 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.805072 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.807724 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:47.807726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:47.807727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:47.807737 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:47.809696 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.810133 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.810135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.810135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.810209 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.810501 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.810501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.810502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.810657 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:47.810734 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:47.811090 (+ 356us) server_negotiation.cc:299] Negotiation successful
0620 08:09:47.811223 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 08:09:47.811913 21346 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38953
I20260620 08:09:47.812032 21346 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:47.812232 21346 heartbeater.cc:511] Master 127.12.132.254:38953 requested a full tablet report, sending...
I20260620 08:09:47.812615 21021 ts_manager.cc:194] Registered new tserver with Master: 3a01918899e84e9081351e6a097510e8 (127.12.132.194:43339)
I20260620 08:09:47.813032 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:51007
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:09:47.835009 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:47.917009 21351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:47.917209 21351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:47.917234 21351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:47.918625 21351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:47.918661 21351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:47.918686 21351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:47.920151 21351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:47.920329 21351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:47.920486 21351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:47.925819 21359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:47.925861 21356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:47.925944 21351 server_base.cc:1034] running on GCE node
W20260620 08:09:47.925844 21357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:47.926227 21351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:47.926421 21351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:47.927556 21351 hybrid_clock.cc:648] HybridClock initialized: now 1781942987927542 us; error 34 us; skew 500 ppm
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:47.929983 21351 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:09:47.930410 21351 webserver.cc:460] Webserver started at http://127.12.132.195:42863/ using document root <none> and password file <none>
I20260620 08:09:47.930593 21351 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:47.930632 21351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:47.930716 21351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:09:47.931667 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "33ea9c91234e43d7a7a882320cabcde8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "4035da3f3981599b03084839b4963118"
server_key_iv: "cc27989153b07243be3c58585bf9725b"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.931945 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "33ea9c91234e43d7a7a882320cabcde8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "4035da3f3981599b03084839b4963118"
server_key_iv: "cc27989153b07243be3c58585bf9725b"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.933279 21351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:09:47.933974 21366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.934136 21351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:47.934202 21351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "33ea9c91234e43d7a7a882320cabcde8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "4035da3f3981599b03084839b4963118"
server_key_iv: "cc27989153b07243be3c58585bf9725b"
server_key_version: "encryptionkey@0"
I20260620 08:09:47.934289 21351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:47.943677 21351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:47.946267 21351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:47.946377 21351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:47.946643 21351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:47.946933 21351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:47.946985 21351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.947034 21351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:47.947062 21351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:47.953151 21351 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:39051
I20260620 08:09:47.953229 21479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:39051 every 8 connection(s)
I20260620 08:09:47.953541 21351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:09:47 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942987, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:09:47.960386 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21351
I20260620 08:09:47.960551 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:09:47.960770 12819 external_mini_cluster.cc:1428] Setting key 6a1ff01513ab73b1292262139ebc1b32
I20260620 08:09:47.963416 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.954269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:33835 (local address 127.12.132.254:38953)
0620 08:09:47.954333 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:47.954335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:47.954564 (+ 229us) server_negotiation.cc:407] Connection header received
0620 08:09:47.955006 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:47.955007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:47.955017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:47.955073 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:47.956063 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.956931 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.958072 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.958252 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.958310 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.960751 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:47.960752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:47.960753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:47.960760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:47.962260 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.962625 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.962626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.962626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.962672 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:47.962874 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:47.962875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:47.962875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:47.963001 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:47.963066 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:47.963292 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 08:09:47.963355 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:09:47.963603 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:47.954252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:33835)
0620 08:09:47.954421 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:47.954468 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:47.954917 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:47.955121 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:47.955124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:47.955672 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:47.955916 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.955922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.957034 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.957035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.957954 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:47.957956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:47.958345 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:47.958346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:47.958466 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:47.959037 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:47.959045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:47.960661 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:09:47.962353 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.962358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.962360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.962561 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.962712 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:09:47.962712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:09:47.962713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:09:47.962794 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:09:47.963070 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:09:47.963074 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:09:47.963177 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 08:09:47.963259 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 08:09:47.963377 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260620 08:09:47.964078 21480 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38953
I20260620 08:09:47.964162 21480 heartbeater.cc:464] Registering TS with master...
I20260620 08:09:47.964287 21480 heartbeater.cc:511] Master 127.12.132.254:38953 requested a full tablet report, sending...
I20260620 08:09:47.964636 21021 ts_manager.cc:194] Registered new tserver with Master: 33ea9c91234e43d7a7a882320cabcde8 (127.12.132.195:39051)
I20260620 08:09:47.965011 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:33835
I20260620 08:09:47.972839 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
Entry for principal oryx/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab.
WARNING: no policy specified for oryx/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
Entry for principal oryx/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab.
WARNING: no policy specified for oryx/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
Entry for principal oryx/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab.
I20260620 08:09:48.042897 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21083
I20260620 08:09:48.047451 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21217
I20260620 08:09:48.052057 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21351
I20260620 08:09:48.056146 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 20989
I20260620 08:09:48.061748 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38953
--webserver_interface=127.12.132.254
--webserver_port=42735
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:09:48.144621 21489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:48.144812 21489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:48.144834 21489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:48.145970 21489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:09:48.146013 21489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:48.146028 21489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:48.146040 21489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:09:48.146052 21489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:09:48.147518 21489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=42735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:48.147754 21489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:48.147995 21489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:48.152923 21497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.152925 21495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.152980 21494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:48.153234 21489 server_base.cc:1034] running on GCE node
I20260620 08:09:48.153544 21489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:48.153827 21489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:48.154994 21489 hybrid_clock.cc:648] HybridClock initialized: now 1781942988154981 us; error 31 us; skew 500 ppm
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942988, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:48.157658 21489 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:09:48.158078 21489 webserver.cc:460] Webserver started at http://127.12.132.254:42735/ using document root <none> and password file <none>
I20260620 08:09:48.158315 21489 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:48.158396 21489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:48.159855 21489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:48.160985 21504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.161250 21489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:48.161373 21489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ef9a22e8035d431b96dd656643fe0d57"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "ea307bcdff14cba2c0e75e0ae928ca58"
server_key_iv: "3646e8289cee565ac7a768d2d81628fd"
server_key_version: "encryptionkey@0"
I20260620 08:09:48.161712 21489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:48.170077 21489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:48.172572 21489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:48.172721 21489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:48.176406 21489 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38953
I20260620 08:09:48.176457 21556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38953 every 8 connection(s)
I20260620 08:09:48.176903 21489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:09:48.178349 21557 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:09:48.178907 21557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Bootstrap starting.
I20260620 08:09:48.187050 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21489
I20260620 08:09:48.187486 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:41593
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:38953
--webserver_port=43995
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:09:48.187824 21557 log.cc:824] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Log is configured to *not* fsync() on all Append() calls
I20260620 08:09:48.189538 21557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:09:48.189819 21557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Bootstrap complete.
I20260620 08:09:48.192556 21557 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:48.192842 21557 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9a22e8035d431b96dd656643fe0d57, State: Initialized, Role: FOLLOWER
I20260620 08:09:48.193216 21557 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:48.193379 21557 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:09:48.193463 21557 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:09:48.193540 21557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:09:48.194955 21557 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:48.195106 21557 leader_election.cc:302] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9a22e8035d431b96dd656643fe0d57; no voters:
I20260620 08:09:48.195333 21557 leader_election.cc:288] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 08:09:48.195427 21561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:09:48.195693 21561 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [term 2 LEADER]: Becoming Leader. State: Replica: ef9a22e8035d431b96dd656643fe0d57, State: Running, Role: LEADER
I20260620 08:09:48.195732 21557 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:09:48.195814 21561 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } }
I20260620 08:09:48.196142 21561 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9a22e8035d431b96dd656643fe0d57. Latest consensus state: current_term: 2 leader_uuid: "ef9a22e8035d431b96dd656643fe0d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } } }
I20260620 08:09:48.196228 21561 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:48.196374 21562 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ef9a22e8035d431b96dd656643fe0d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9a22e8035d431b96dd656643fe0d57" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38953 } } }
I20260620 08:09:48.196447 21562 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57 [sys.catalog]: This master's current role is: LEADER
I20260620 08:09:48.196830 21567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:09:48.197364 21567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:09:48.197525 21567 catalog_manager.cc:1259] Loaded cluster ID: 213444feb8074a419e59469a33449bde
I20260620 08:09:48.197785 21567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:09:48.198940 21567 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:09:48.199122 21567 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef9a22e8035d431b96dd656643fe0d57: Loaded TSK: 0
I20260620 08:09:48.199427 21567 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:09:48.275038 21560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:48.275235 21560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:48.275259 21560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:48.276899 21560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:48.276942 21560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:48.276970 21560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:09:48.278489 21560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.193.keytab
--principal=oryx/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:41593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=43995
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:48.278716 21560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:48.278963 21560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:48.283874 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.283911 21583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:48.284049 21560 server_base.cc:1034] running on GCE node
W20260620 08:09:48.284148 21584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:48.284339 21560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:48.284605 21560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:48.285780 21560 hybrid_clock.cc:648] HybridClock initialized: now 1781942988285766 us; error 30 us; skew 500 ppm
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942988, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:48.288357 21560 init.cc:377] Logged in from keytab as oryx/127.12.132.193@KRBTEST.COM (short username oryx)
I20260620 08:09:48.288811 21560 webserver.cc:460] Webserver started at http://127.12.132.193:43995/ using document root <none> and password file <none>
I20260620 08:09:48.289036 21560 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:48.289271 21560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:48.291229 21560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:09:48.291996 21593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.292220 21560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:48.292321 21560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "2ec218f5d2eb4843a7df3aef0f76c5a7"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "cc231435123e7d1328c0cfa37a8ec130"
server_key_iv: "b6ba9ce9f9e6c2fb5adac9265bf81720"
server_key_version: "encryptionkey@0"
I20260620 08:09:48.292788 21560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:48.308646 21560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:48.310839 21560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:48.311015 21560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:48.311296 21560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:48.311674 21560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:48.311733 21560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.311772 21560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:48.311802 21560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.320222 21560 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:41593
I20260620 08:09:48.320289 21706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:41593 every 8 connection(s)
I20260620 08:09:48.320654 21560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:09:48.324447 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21560
I20260620 08:09:48.324910 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:43339
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:38953
--webserver_port=39829
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.331768 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.321415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:36437)
0620 08:09:48.321607 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.321654 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.322130 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.322651 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.322655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.323450 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.323708 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.323714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.325845 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.325846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.327126 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.327129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.327968 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.327970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.328123 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.328741 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.328751 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.331557 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
W20260620 08:09:48.331990 21707 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:38953 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.132.254:38953: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
W20260620 08:09:48.332216 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.321477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36437 (local address 127.12.132.254:38953)
0620 08:09:48.322282 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.322289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.322310 (+ 21us) server_negotiation.cc:407] Connection header received
0620 08:09:48.322421 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.322426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.322463 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.322556 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.323934 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.325672 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.327244 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.327861 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.327965 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.331728 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:36437: BlockingRecv error: recv got EOF from 127.12.132.193:36437 (error 108)
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":99,"threads_started":1}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.337646 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.332212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:39551)
0620 08:09:48.332396 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.332409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.332459 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.332644 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.332646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.332825 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.332886 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.332888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.333710 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.333712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.334632 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.334635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.335121 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.335122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.335197 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.335628 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.335635 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.337560 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 08:09:48.337708 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.332318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:39551 (local address 127.12.132.254:38953)
0620 08:09:48.332354 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.332357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.332529 (+ 172us) server_negotiation.cc:407] Connection header received
0620 08:09:48.332554 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.332555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.332568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.332616 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.332973 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.333622 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.334728 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.335028 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.335099 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.337643 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:39551: BlockingRecv error: recv got EOF from 127.12.132.193:39551 (error 108)
Metrics: {"server-negotiator.queue_time_us":21,"thread_start_us":112,"threads_started":1}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.344602 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.337921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:53557)
0620 08:09:48.340093 (+ 2172us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.340105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.340164 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.340325 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.340326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.340476 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.340525 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.340527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.341104 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.341104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.341919 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.341921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.342277 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.342278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.342331 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.342718 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.342723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.344534 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2153}
W20260620 08:09:48.344736 21707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:09:48.344851 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.337989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:53557 (local address 127.12.132.254:38953)
0620 08:09:48.338044 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.338045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.340212 (+ 2167us) server_negotiation.cc:407] Connection header received
0620 08:09:48.340238 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.340239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.340250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.340306 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.340591 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.341033 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.342030 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.342205 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.342261 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.344620 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:53557: BlockingRecv error: recv got EOF from 127.12.132.193:53557 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260620 08:09:48.413064 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:48.413300 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:48.413334 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:48.414747 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:48.414809 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:48.414855 21711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:09:48.416724 21711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.194.keytab
--principal=oryx/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:43339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=39829
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:48.417127 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:48.417423 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:48.422861 21720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.422861 21718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.422907 21717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:48.423094 21711 server_base.cc:1034] running on GCE node
I20260620 08:09:48.423338 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:48.423585 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:48.424791 21711 hybrid_clock.cc:648] HybridClock initialized: now 1781942988424781 us; error 41 us; skew 500 ppm
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942988, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:48.427242 21711 init.cc:377] Logged in from keytab as oryx/127.12.132.194@KRBTEST.COM (short username oryx)
I20260620 08:09:48.427697 21711 webserver.cc:460] Webserver started at http://127.12.132.194:39829/ using document root <none> and password file <none>
I20260620 08:09:48.427932 21711 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:48.428015 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:48.429412 21711 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:48.429986 21727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.430209 21711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:09:48.430307 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "3a01918899e84e9081351e6a097510e8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "e88827eaba956510a3e4a2af5feac525"
server_key_iv: "26f25773f9bbcbdd192426a532b54bc3"
server_key_version: "encryptionkey@0"
I20260620 08:09:48.430662 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:48.449826 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:48.452423 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:48.452847 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:48.453204 21711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:48.453670 21711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:48.453734 21711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.453787 21711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:48.453835 21711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.460816 21711 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:43339
I20260620 08:09:48.460853 21840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:43339 every 8 connection(s)
I20260620 08:09:48.461222 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:09:48.462072 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21711
I20260620 08:09:48.462393 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:39051
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:38953
--webserver_port=42863
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 08:09:48.469734 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.461976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:54653 (local address 127.12.132.254:38953)
0620 08:09:48.462029 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.462031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.462339 (+ 308us) server_negotiation.cc:407] Connection header received
0620 08:09:48.462997 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.462998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.463015 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.463060 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.464081 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.464788 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.465845 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.466191 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.466266 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.469685 (+ 3419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:54653: BlockingRecv error: recv got EOF from 127.12.132.194:54653 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:09:48.469815 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.461990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:54653)
0620 08:09:48.462198 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.462245 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.462874 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.463126 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.463130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.463771 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.463976 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.463980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.464888 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.464889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.465755 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.465758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.466296 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.466297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.466415 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.466992 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.467025 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.469545 (+ 2520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
W20260620 08:09:48.470093 21841 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:38953 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.132.254:38953: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.475116 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.470321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:43007)
0620 08:09:48.470380 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.470391 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.470432 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.470577 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.470578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.470707 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.470751 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.470752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.471745 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.471746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.472261 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.472263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.472770 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.472771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.472823 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.473297 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.473302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.475046 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 08:09:48.475203 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.470399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:43007 (local address 127.12.132.254:38953)
0620 08:09:48.470448 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.470451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.470459 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:09:48.470489 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.470490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.470501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.470569 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.470840 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.471628 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.472346 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.472694 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.472773 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.475127 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:43007: BlockingRecv error: recv got EOF from 127.12.132.194:43007 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.479967 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.475593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:48735)
0620 08:09:48.475639 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.475653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.475705 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.475854 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.475855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.475982 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.476013 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.476014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.476683 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.476684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.477163 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.477165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.477543 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.477544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.477597 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.478077 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.478082 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.479891 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 08:09:48.480090 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.475649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:48735 (local address 127.12.132.254:38953)
0620 08:09:48.475710 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.475712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.475745 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:09:48.475772 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.475773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.475781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.475837 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.476104 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.476597 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.477258 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.477438 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.477534 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.479995 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:48735: BlockingRecv error: recv got EOF from 127.12.132.194:48735 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 08:09:48.480127 21841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:09:48.548379 21844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:09:48.548599 21844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:09:48.548621 21844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:09:48.549696 21844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:09:48.549728 21844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:09:48.549772 21844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:09:48.551136 21844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/krb5kdc/oryx_127.12.132.195.keytab
--principal=oryx/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:39051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=42863
--tserver_master_addrs=127.12.132.254:38953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:09:48.551308 21844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:09:48.551468 21844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:09:48.556478 21849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.556582 21852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:09:48.556486 21850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:09:48.556605 21844 server_base.cc:1034] running on GCE node
I20260620 08:09:48.556785 21844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:09:48.556978 21844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:09:48.558138 21844 hybrid_clock.cc:648] HybridClock initialized: now 1781942988558117 us; error 39 us; skew 500 ppm
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781942988, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:09:48.560722 21844 init.cc:377] Logged in from keytab as oryx/127.12.132.195@KRBTEST.COM (short username oryx)
I20260620 08:09:48.561267 21844 webserver.cc:460] Webserver started at http://127.12.132.195:42863/ using document root <none> and password file <none>
I20260620 08:09:48.561520 21844 fs_manager.cc:359] Metadata directory not provided
I20260620 08:09:48.561602 21844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:09:48.562944 21844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:09:48.563542 21859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.563710 21844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 08:09:48.563800 21844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "33ea9c91234e43d7a7a882320cabcde8"
format_stamp: "Formatted at 2026-06-20 08:09:47 on dist-test-slave-z0sp"
server_key: "4035da3f3981599b03084839b4963118"
server_key_iv: "cc27989153b07243be3c58585bf9725b"
server_key_version: "encryptionkey@0"
I20260620 08:09:48.564131 21844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:09:48.579528 21844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:09:48.581800 21844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:09:48.581966 21844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:09:48.582206 21844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:09:48.582510 21844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:09:48.582563 21844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.582597 21844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:09:48.582624 21844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:09:48.588662 21844 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:39051
I20260620 08:09:48.588721 21972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:39051 every 8 connection(s)
I20260620 08:09:48.589071 21844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.597580 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.589734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:51089)
0620 08:09:48.589895 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.589935 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.590388 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.590644 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.590647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.591194 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.591381 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.591385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.592594 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.592596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.593689 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.593693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.594218 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.594220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.594355 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.594951 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.594961 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.597444 (+ 2483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
W20260620 08:09:48.597594 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.589772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:51089 (local address 127.12.132.254:38953)
0620 08:09:48.589823 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.589825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.590026 (+ 201us) server_negotiation.cc:407] Connection header received
0620 08:09:48.590500 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.590501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.590513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.590574 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.591582 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.592286 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.593827 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.594034 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.594154 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.597547 (+ 3393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:51089: BlockingRecv error: recv got EOF from 127.12.132.195:51089 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 08:09:48.597782 21973 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:38953 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.132.254:38953: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
I20260620 08:09:48.598129 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 21844
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
W20260620 08:09:48.603807 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.598230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:43243 (local address 127.12.132.254:38953)
0620 08:09:48.598275 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.598277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.598286 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:48.598309 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.598310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.598321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.598385 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.598909 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.599729 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.600820 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.601135 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.601238 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.603757 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:43243: BlockingRecv error: recv got EOF from 127.12.132.195:43243 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:09:48.603737 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.597996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:43243)
0620 08:09:48.598051 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.598064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.598114 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.598451 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.598453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.598763 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.598830 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.598834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.599851 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.599852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.600708 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.600710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.601279 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.601280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.601342 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.601916 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.601921 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.603666 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 08:09:48.607278 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.598760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.12.132.254:38953)
0620 08:09:48.598851 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.598853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.599002 (+ 149us) server_negotiation.cc:407] Connection header received
0620 08:09:48.599042 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.599043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.599052 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.599110 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.599851 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.600611 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.601888 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.602209 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.602297 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.603647 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:09:48.603652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:09:48.603652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:09:48.603665 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:09:48.605449 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:48.605841 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:48.605844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:48.605844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:48.605955 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:09:48.606210 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:09:48.606210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:09:48.606211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:09:48.606561 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:09:48.606666 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 08:09:48.607046 (+ 380us) server_negotiation.cc:299] Negotiation successful
0620 08:09:48.607176 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:48 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:48.609185 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:48.604321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:36349)
0620 08:09:48.604392 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:48.604405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:48.604450 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:48.604858 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:48.604859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:48.604988 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:48.605012 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.605014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.605670 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.605671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.606366 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:48.606369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.606985 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.606986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:48.607037 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.607367 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:48.607372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:48.609144 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260620 08:09:48.609258 21710 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:48.604427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:36349 (local address 127.12.132.254:38953)
0620 08:09:48.604508 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:48.604510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:48.604519 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:48.604538 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:48.604539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:48.604547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:48.604850 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:48.605105 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.605592 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.606454 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:48.606778 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:48.606850 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:48.609221 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:36349: BlockingRecv error: recv got EOF from 127.12.132.195:36349 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 08:09:48.609405 21973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:49.349929 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:49.345175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:52569)
0620 08:09:49.345316 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:49.345335 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:49.345385 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:49.345522 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:49.345523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:49.345693 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:49.345742 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.345744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.346420 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.346421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.346972 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.346974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.347581 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.347582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.347627 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.348130 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:49.348134 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:49.349854 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
W20260620 08:09:49.350099 21978 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:49.345246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52569 (local address 127.12.132.254:38953)
0620 08:09:49.345383 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:49.345385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:49.345395 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:49.345431 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:49.345431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:49.345441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:49.345508 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:49.345825 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.346339 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.347172 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.347455 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.347583 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.349990 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:52569: BlockingRecv error: recv got EOF from 127.12.132.193:52569 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:49.486449 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:49.480606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:42159)
0620 08:09:49.480821 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:49.480836 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:49.480976 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:49.481170 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:49.481171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:49.481400 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:49.481475 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.481477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.482233 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.482234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.482825 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.482828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.483211 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.483212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.483276 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.483848 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:49.483854 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:49.486351 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
W20260620 08:09:49.486543 21978 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:49.480711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:42159 (local address 127.12.132.254:38953)
0620 08:09:49.480815 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:49.480818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:49.480937 (+ 119us) server_negotiation.cc:407] Connection header received
0620 08:09:49.481056 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:49.481058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:49.481077 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:49.481141 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:49.481564 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.482142 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.482916 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.483120 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.483210 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.486478 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:42159: BlockingRecv error: recv got EOF from 127.12.132.194:42159 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:49 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:49.614792 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:49.609785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:34277)
0620 08:09:49.609950 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:49.609962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:49.610049 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:49.610237 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:49.610238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:49.610416 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:49.610460 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.610462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.611131 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.611132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.611670 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:49.611672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.612032 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.612033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:49.612088 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.612612 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:49.612616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:49.614720 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
W20260620 08:09:49.614969 21978 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:49.609957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:34277 (local address 127.12.132.254:38953)
0620 08:09:49.610023 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:49.610026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:49.610036 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:49.610137 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:49.610138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:49.610159 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:49.610215 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:49.610539 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.611049 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.611756 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:49.611948 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:49.612032 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:49.614899 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:34277: BlockingRecv error: recv got EOF from 127.12.132.195:34277 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:50.355618 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:50.350480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:37979)
0620 08:09:50.350638 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:50.350651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:50.350723 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:50.350920 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:50.350921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:50.351085 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:50.351131 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.351133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.351892 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.351893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.352570 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.352573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.352938 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.352939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.352986 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.353566 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:50.353577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:50.355561 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":54,"threads_started":1}
W20260620 08:09:50.355778 21982 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:50.350613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:37979 (local address 127.12.132.254:38953)
0620 08:09:50.350786 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:50.350788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:50.350797 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:50.350820 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:50.350820 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:50.350830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:50.350896 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:50.351198 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.351830 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.352646 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.352846 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.352910 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.355661 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:37979: BlockingRecv error: recv got EOF from 127.12.132.193:37979 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:50.492249 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:50.487025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:37041)
0620 08:09:50.487175 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:50.487185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:50.487242 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:50.487387 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:50.487387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:50.487547 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:50.487600 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.487601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.488477 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.488479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.489133 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.489136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.489672 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.489673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.489743 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.490279 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:50.490284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:50.492187 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260620 08:09:50.492362 21982 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:50.487099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:37041 (local address 127.12.132.254:38953)
0620 08:09:50.487188 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:50.487190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:50.487229 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:09:50.487288 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:50.487288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:50.487299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:50.487365 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:50.487742 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.488329 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.489290 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.489561 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.489638 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.492285 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:37041: BlockingRecv error: recv got EOF from 127.12.132.194:37041 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:50 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:50.620680 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:50.615269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:34767)
0620 08:09:50.615476 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:50.615496 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:50.615682 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:50.615906 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:50.615908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:50.616072 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:50.616123 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.616124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.616932 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.616933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.617477 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:50.617479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.618033 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.618035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:50.618096 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.618653 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:50.618659 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:50.620620 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":57,"threads_started":1}
W20260620 08:09:50.620822 21982 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:50.615591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:34767 (local address 127.12.132.254:38953)
0620 08:09:50.615690 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:50.615692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:50.615702 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:09:50.615752 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:50.615756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:50.615768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:50.615820 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:50.616200 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.616727 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.617595 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:50.617772 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:50.617975 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:50.620723 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:34767: BlockingRecv error: recv got EOF from 127.12.132.195:34767 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:51.361549 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:51.356161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:43985)
0620 08:09:51.356325 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:51.356336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:51.356398 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:51.356768 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:51.356769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:51.356921 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:51.356979 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.356981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.357847 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.357848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.358475 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.358479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.358975 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.358977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.359047 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.359580 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:51.359590 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:51.361450 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":118,"threads_started":1}
W20260620 08:09:51.361614 21986 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:51.356386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:43985 (local address 127.12.132.254:38953)
0620 08:09:51.356584 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:51.356586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:51.356598 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:51.356631 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:51.356631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:51.356647 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:51.356749 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:51.357116 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.357705 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.358595 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.358852 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.358925 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.361529 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:43985: BlockingRecv error: recv got EOF from 127.12.132.193:43985 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:51.497556 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:51.492822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:40869)
0620 08:09:51.492949 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:51.492966 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:51.493046 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:51.493167 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:51.493168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:51.493358 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:51.493407 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.493409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.494108 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.494109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.494632 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.494633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.495075 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.495076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.495119 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.495615 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:51.495620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:51.497471 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
W20260620 08:09:51.497778 21986 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:51.492858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40869 (local address 127.12.132.254:38953)
0620 08:09:51.492903 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:51.492905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:51.493045 (+ 140us) server_negotiation.cc:407] Connection header received
0620 08:09:51.493105 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:51.493106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:51.493118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:51.493160 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:51.493525 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.494029 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.494705 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.494964 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.495059 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.497734 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:40869: BlockingRecv error: recv got EOF from 127.12.132.194:40869 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:51 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:51.626528 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:51.621324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:38169)
0620 08:09:51.621548 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:51.621568 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:51.621633 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:51.621942 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:51.621943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:51.622171 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:51.622221 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.622223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.622987 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.622988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.623502 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:51.623503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.623877 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.623878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:51.623935 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.624430 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:51.624435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:51.626465 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":154,"threads_started":1}
W20260620 08:09:51.626658 21986 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:51.621511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:38169 (local address 127.12.132.254:38953)
0620 08:09:51.621606 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:51.621609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:51.621679 (+ 70us) server_negotiation.cc:407] Connection header received
0620 08:09:51.621722 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:51.621723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:51.621741 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:51.621812 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:51.622367 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.622896 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.623605 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:51.623788 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:51.623874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:51.626575 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:38169: BlockingRecv error: recv got EOF from 127.12.132.195:38169 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:52.368511 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:52.362103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:37211)
0620 08:09:52.362279 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:52.362291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:52.362345 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:52.362484 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:52.362485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:52.362661 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:52.362708 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.362709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.363697 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.363698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.364348 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.364351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.364849 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.364850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.364898 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.365436 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:52.365440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:52.368383 (+ 2943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
W20260620 08:09:52.368556 21990 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:52.362227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:37211 (local address 127.12.132.254:38953)
0620 08:09:52.362361 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:52.362363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:52.362372 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:52.362393 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:52.362394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:52.362404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:52.362469 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:52.362943 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.363539 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.364472 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.364743 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.364812 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.368462 (+ 3650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:37211: BlockingRecv error: recv got EOF from 127.12.132.193:37211 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:52.510059 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:52.498090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:40059)
0620 08:09:52.498243 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:52.498255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:52.498305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:52.498549 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:52.498552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:52.498929 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:52.498983 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.498986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.499813 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.499814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.500335 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.500337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.500761 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.500762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.500828 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.507505 (+ 6677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:52.507513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:52.509955 (+ 2442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260620 08:09:52.510133 21990 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:52.498208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:40059 (local address 127.12.132.254:38953)
0620 08:09:52.498276 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:52.498278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:52.498291 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:09:52.498367 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:52.498367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:52.498380 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:52.498436 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:52.499092 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.499712 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.500424 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.500673 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.500761 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.510066 (+ 9305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:40059: BlockingRecv error: recv got EOF from 127.12.132.194:40059 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:52 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:52.632349 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:52.627186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:50695)
0620 08:09:52.627378 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:52.627390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:52.627470 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:52.627685 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:52.627686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:52.627889 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:52.627957 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.627959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.628690 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.628691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.629400 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:52.629405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.629788 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.629789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:52.629841 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.630384 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:52.630389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:52.632291 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
W20260620 08:09:52.632719 21990 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:52.627287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:50695 (local address 127.12.132.254:38953)
0620 08:09:52.627331 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:52.627333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:52.627489 (+ 156us) server_negotiation.cc:407] Connection header received
0620 08:09:52.627522 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:52.627522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:52.627538 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:52.627621 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:52.628059 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.628582 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.629500 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:52.629688 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:52.629755 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:52.632610 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:50695: BlockingRecv error: recv got EOF from 127.12.132.195:50695 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:53.374943 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:53.369312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:41827)
0620 08:09:53.369571 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:53.369584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:53.369691 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:53.369923 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:53.369925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:53.370160 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:53.370230 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.370232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.371090 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.371094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.371702 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.371706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.372267 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.372269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.372351 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.372888 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:53.372894 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:53.374861 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":147,"threads_started":1}
W20260620 08:09:53.375089 21993 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:53.369360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:41827 (local address 127.12.132.254:38953)
0620 08:09:53.369551 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:53.369553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:53.369667 (+ 114us) server_negotiation.cc:407] Connection header received
0620 08:09:53.369793 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:53.369794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:53.369817 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:53.369915 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:53.370321 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.370931 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.371925 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.372154 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.372244 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.374991 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:41827: BlockingRecv error: recv got EOF from 127.12.132.193:41827 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:53.515257 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:53.510603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:56931)
0620 08:09:53.510791 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:53.510801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:53.510855 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:53.511022 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:53.511023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:53.511187 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:53.511235 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.511237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.511888 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.511889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.512409 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.512410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.512857 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.512858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.512903 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.513364 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:53.513368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:53.515155 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
W20260620 08:09:53.515298 21993 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:53.510702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:56931 (local address 127.12.132.254:38953)
0620 08:09:53.510756 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:53.510759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:53.510909 (+ 150us) server_negotiation.cc:407] Connection header received
0620 08:09:53.510936 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:53.510937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:53.510949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:53.511000 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:53.511302 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.511811 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.512516 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.512723 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.512780 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.515234 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:56931: BlockingRecv error: recv got EOF from 127.12.132.194:56931 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:53 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:53.638091 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:53.632890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:41619)
0620 08:09:53.633056 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:53.633069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:53.633167 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:53.633379 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:53.633380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:53.633557 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:53.633604 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.633605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.634351 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.634353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.634898 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:53.634900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.635284 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.635285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:53.635358 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.636064 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:53.636069 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:53.638031 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
W20260620 08:09:53.638244 21993 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:53.632987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:41619 (local address 127.12.132.254:38953)
0620 08:09:53.633057 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:53.633060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:53.633146 (+ 86us) server_negotiation.cc:407] Connection header received
0620 08:09:53.633247 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:53.633248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:53.633267 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:53.633327 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:53.633688 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.634264 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.634988 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:53.635194 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:53.635282 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:53.638164 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:41619: BlockingRecv error: recv got EOF from 127.12.132.195:41619 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:54.381002 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:54.375513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:55681)
0620 08:09:54.375684 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:54.375695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:54.375754 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:54.375889 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:54.375890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:54.376053 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:54.376103 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.376105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.376761 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.376762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.377275 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.377277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.377621 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.377621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.377813 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.378295 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:54.378300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:54.380939 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
W20260620 08:09:54.381270 21998 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:54.375615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:55681 (local address 127.12.132.254:38953)
0620 08:09:54.375744 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:54.375746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:54.375755 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:54.375802 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:54.375803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:54.375813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:54.375870 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:54.376166 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.376688 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.377354 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.377553 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.377621 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.381176 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:55681: BlockingRecv error: recv got EOF from 127.12.132.193:55681 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:54.521042 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:54.515986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:35723)
0620 08:09:54.516174 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:54.516188 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:54.516307 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:54.516515 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:54.516516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:54.516701 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:54.516744 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.516746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.517431 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.517432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.517980 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.517981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.518346 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.518347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.518411 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.518891 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:54.518898 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:54.520982 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
W20260620 08:09:54.521207 21998 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:54.516091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:35723 (local address 127.12.132.254:38953)
0620 08:09:54.516177 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:54.516180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:54.516266 (+ 86us) server_negotiation.cc:407] Connection header received
0620 08:09:54.516403 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:54.516404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:54.516425 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:54.516486 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:54.516829 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.517339 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.518070 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.518264 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.518338 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.521119 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:35723: BlockingRecv error: recv got EOF from 127.12.132.194:35723 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:54 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:54.643334 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:54.638645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:32869)
0620 08:09:54.638789 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:54.638799 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:54.638881 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:54.639069 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:54.639070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:54.639247 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:54.639291 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.639292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.639950 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.639952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.640466 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:54.640467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.640880 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.640881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:54.640946 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.641454 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:54.641459 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:54.643276 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
W20260620 08:09:54.643493 21998 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:54.638791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:32869 (local address 127.12.132.254:38953)
0620 08:09:54.638880 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:54.638883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:54.638892 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:54.638942 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:54.638944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:54.638958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:54.639062 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:54.639373 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.639871 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.640601 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:54.640793 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:54.640878 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:54.643409 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:32869: BlockingRecv error: recv got EOF from 127.12.132.195:32869 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:55.387125 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:55.381595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:43345)
0620 08:09:55.381739 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:55.381758 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:55.381810 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:55.382208 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:55.382209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:55.382368 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:55.382420 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.382421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.383214 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.383216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.383828 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.383830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.384288 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.384290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.384354 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.384955 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:55.384959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:55.387063 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
W20260620 08:09:55.387420 22002 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:55.381718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:43345 (local address 127.12.132.254:38953)
0620 08:09:55.381996 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:55.381999 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:55.382015 (+ 16us) server_negotiation.cc:407] Connection header received
0620 08:09:55.382064 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:55.382064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:55.382082 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:55.382183 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:55.382561 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.383090 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.383951 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.384185 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.384252 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.387326 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:43345: BlockingRecv error: recv got EOF from 127.12.132.193:43345 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":209,"threads_started":1}
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:55.526960 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:55.521691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:54969)
0620 08:09:55.521948 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:55.521964 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:55.522045 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:55.522193 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:55.522194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:55.522384 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:55.522428 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.522430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.523134 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.523135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.523677 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.523678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.524058 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.524059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.524114 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.524602 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:55.524607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:55.526862 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":130,"threads_started":1}
W20260620 08:09:55.527046 22002 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:55.521825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:54969 (local address 127.12.132.254:38953)
0620 08:09:55.521942 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:55.521944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:55.522027 (+ 83us) server_negotiation.cc:407] Connection header received
0620 08:09:55.522106 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:55.522107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:55.522121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:55.522177 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:55.522533 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.523047 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.523763 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.523968 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.524057 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.526944 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:54969: BlockingRecv error: recv got EOF from 127.12.132.194:54969 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:55 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:55.649156 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:55.643928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:53329)
0620 08:09:55.644085 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:55.644096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:55.644148 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:55.644392 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:55.644393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:55.644622 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:55.644675 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.644677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.645439 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.645441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.646057 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:55.646058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.646500 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.646501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:55.646567 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.647143 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:55.647148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:55.649094 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260620 08:09:55.649259 22002 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:55.644031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:53329 (local address 127.12.132.254:38953)
0620 08:09:55.644211 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:55.644213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:55.644229 (+ 16us) server_negotiation.cc:407] Connection header received
0620 08:09:55.644272 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:55.644273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:55.644292 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:55.644369 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:55.644813 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.645318 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.646204 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:55.646391 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:55.646450 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:55.649187 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:53329: BlockingRecv error: recv got EOF from 127.12.132.195:53329 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:56.393577 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:56.387705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:44443)
0620 08:09:56.387889 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:56.387904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:56.387971 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:56.388342 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:56.388344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:56.388635 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:56.388707 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.388710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.389461 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.389462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.390164 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.390168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.390612 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.390613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.390668 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.391289 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:56.391299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:56.393459 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
W20260620 08:09:56.393707 22006 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:56.387878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:44443 (local address 127.12.132.254:38953)
0620 08:09:56.388099 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:56.388101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:56.388115 (+ 14us) server_negotiation.cc:407] Connection header received
0620 08:09:56.388147 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:56.388148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:56.388163 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:56.388258 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:56.388832 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.389358 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.390325 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.390522 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.390581 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.393605 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:44443: BlockingRecv error: recv got EOF from 127.12.132.193:44443 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:56.532245 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:56.527596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:42499)
0620 08:09:56.527753 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:56.527772 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:56.527822 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:56.527981 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:56.527982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:56.528143 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:56.528191 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.528192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.529022 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.529023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.529534 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.529535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.529899 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.529900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.529942 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.530392 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:56.530397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:56.532177 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
W20260620 08:09:56.532444 22006 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:56.527669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:42499 (local address 127.12.132.254:38953)
0620 08:09:56.527750 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:56.527752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:56.527852 (+ 100us) server_negotiation.cc:407] Connection header received
0620 08:09:56.527891 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:56.527892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:56.527904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:56.527961 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:56.528258 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.528926 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.529635 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.529827 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.529892 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.532344 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:42499: BlockingRecv error: recv got EOF from 127.12.132.194:42499 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:56 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:56.654922 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:56.649815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:33239)
0620 08:09:56.650017 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:56.650033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:56.650131 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:56.650399 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:56.650400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:56.650568 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:56.650719 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.650724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.651443 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.651444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.652076 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:56.652079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.652439 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.652440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:56.652480 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.652976 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:56.652980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:56.654787 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":127,"threads_started":1}
W20260620 08:09:56.654994 22006 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:56.649939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:33239 (local address 127.12.132.254:38953)
0620 08:09:56.650018 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:56.650021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:56.650158 (+ 137us) server_negotiation.cc:407] Connection header received
0620 08:09:56.650195 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:56.650196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:56.650211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:56.650293 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:56.650856 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.651344 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.652168 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:56.652359 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:56.652425 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:56.654921 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:33239: BlockingRecv error: recv got EOF from 127.12.132.195:33239 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:57.399094 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:57.394093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:37289)
0620 08:09:57.394237 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:57.394255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:57.394305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:57.394482 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:57.394484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:57.394656 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:57.394705 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.394707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.395400 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.395401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.396050 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.396052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.396425 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.396426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.396535 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.397046 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:57.397051 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:57.399032 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260620 08:09:57.399227 22010 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:57.394175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:37289 (local address 127.12.132.254:38953)
0620 08:09:57.394325 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:57.394327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:57.394336 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:57.394358 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:57.394358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:57.394370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:57.394438 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:57.394784 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.395285 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.396128 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.396322 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.396379 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.399135 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:37289: BlockingRecv error: recv got EOF from 127.12.132.193:37289 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:57.541723 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:57.532823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:44709)
0620 08:09:57.533030 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:57.533050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:57.533173 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:57.533342 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:57.533343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:57.533512 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:57.533560 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.533562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.534362 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.534363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.534877 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.534879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.535373 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.535374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.535413 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.539622 (+ 4209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:57.539627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:57.541661 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
W20260620 08:09:57.541891 22010 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:57.533000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:44709 (local address 127.12.132.254:38953)
0620 08:09:57.533104 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:57.533105 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:57.533147 (+ 42us) server_negotiation.cc:407] Connection header received
0620 08:09:57.533245 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:57.533246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:57.533258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:57.533327 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:57.533675 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.534288 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.535010 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.535296 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.535367 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.541793 (+ 6426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:44709: BlockingRecv error: recv got EOF from 127.12.132.194:44709 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:57 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:57.660315 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:57.655443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:49485)
0620 08:09:57.655618 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:57.655631 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:57.655697 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:57.655869 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:57.655870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:57.656062 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:57.656109 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.656111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.656859 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.656860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.657387 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:57.657388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.657756 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.657757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:57.657819 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.658334 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:57.658339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:57.660250 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
W20260620 08:09:57.660534 22010 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:57.655567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:49485 (local address 127.12.132.254:38953)
0620 08:09:57.655673 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:57.655675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:57.655687 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:09:57.655774 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:57.655775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:57.655797 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:57.655864 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:57.656203 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.656777 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.657487 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:57.657671 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:57.657755 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:57.660396 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:49485: BlockingRecv error: recv got EOF from 127.12.132.195:49485 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:58.404783 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:58.399610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:37395)
0620 08:09:58.399753 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:58.399763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:58.399826 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:58.399990 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:58.399990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:58.400150 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:58.400198 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.400200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.400837 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.400838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.401352 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.401353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.401726 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.401727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.401766 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.402288 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:58.402310 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:58.404707 (+ 2397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
W20260620 08:09:58.405177 22014 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:58.399728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:37395 (local address 127.12.132.254:38953)
0620 08:09:58.399861 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:58.399863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:58.399872 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:09:58.399897 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:58.399898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:58.399908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:58.399967 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:58.400264 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.400777 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.401450 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.401651 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.401713 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.404958 (+ 3245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:37395: BlockingRecv error: recv got EOF from 127.12.132.193:37395 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:58.547917 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:58.542406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:56973)
0620 08:09:58.542588 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:58.542603 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:58.542694 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:58.542958 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:58.542960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:58.543138 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:58.543193 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.543195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.543938 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.543939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.544624 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.544627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.545204 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.545205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.545281 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.545763 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:58.545768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:58.547828 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
W20260620 08:09:58.548161 22014 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:58.542485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:56973 (local address 127.12.132.254:38953)
0620 08:09:58.542565 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:58.542568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:58.542694 (+ 126us) server_negotiation.cc:407] Connection header received
0620 08:09:58.542751 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:58.542752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:58.542763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:58.542888 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:58.543319 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.543824 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.544777 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.545098 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.545192 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.548085 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:56973: BlockingRecv error: recv got EOF from 127.12.132.194:56973 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:58 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:58.666386 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:58.660854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:54853)
0620 08:09:58.661091 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:58.661118 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:58.661241 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:58.661457 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:58.661459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:58.661626 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:58.661681 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.661683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.662343 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.662344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.662856 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:58.662857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.663216 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.663217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:58.663294 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.663938 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:58.663943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:58.666284 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":69,"threads_started":1}
W20260620 08:09:58.666574 22014 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:58.661114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:54853 (local address 127.12.132.254:38953)
0620 08:09:58.661206 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:58.661207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:58.661218 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:09:58.661313 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:58.661314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:58.661328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:58.661387 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:58.661781 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.662272 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.662924 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:58.663097 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:58.663152 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:58.666498 (+ 3346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:54853: BlockingRecv error: recv got EOF from 127.12.132.195:54853 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:59.410218 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:59.405344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:48481)
0620 08:09:59.405634 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:59.405646 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:59.405700 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:59.405834 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:59.405834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:59.406023 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:59.406071 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.406073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.406699 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.406700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.407250 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.407252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.407572 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.407573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.407610 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.408081 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:59.408086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:59.410112 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":202,"threads_started":1}
W20260620 08:09:59.410266 22018 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:59.405561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:48481 (local address 127.12.132.254:38953)
0620 08:09:59.405662 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:59.405666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:59.405683 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:09:59.405747 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:59.405747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:59.405758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:59.405813 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:59.406140 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.406625 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.407324 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.407509 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.407562 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.410209 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:48481: BlockingRecv error: recv got EOF from 127.12.132.193:48481 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":47,"threads_started":1}
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:59.553694 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:59.548577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:35613)
0620 08:09:59.548812 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:59.548828 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:59.548943 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:59.549110 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:59.549111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:59.549304 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:59.549349 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.549350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.550035 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.550036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.550671 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.550673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.551052 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.551053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.551112 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.551568 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:59.551573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:59.553584 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":129,"threads_started":1}
W20260620 08:09:59.553730 22018 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:59.548731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:35613 (local address 127.12.132.254:38953)
0620 08:09:59.548817 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:59.548820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:59.548919 (+ 99us) server_negotiation.cc:407] Connection header received
0620 08:09:59.549023 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:59.549024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:59.549038 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:59.549100 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:59.549440 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.549953 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.550758 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.550967 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.551040 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.553664 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:35613: BlockingRecv error: recv got EOF from 127.12.132.194:35613 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:09:59 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:09:59.672477 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:09:59.666920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:52699)
0620 08:09:59.667146 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0620 08:09:59.667163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 08:09:59.667300 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:09:59.667481 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:09:59.667482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:09:59.667747 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:09:59.667815 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.667816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.668615 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.668617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.669207 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:09:59.669209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.669607 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.669608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:09:59.669664 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.670330 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:09:59.670336 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:09:59.672404 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":60,"threads_started":1}
W20260620 08:09:59.672821 22018 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:09:59.667056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:52699 (local address 127.12.132.254:38953)
0620 08:09:59.667151 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 08:09:59.667153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:09:59.667271 (+ 118us) server_negotiation.cc:407] Connection header received
0620 08:09:59.667367 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:09:59.667368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:09:59.667390 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:09:59.667475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:09:59.667920 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.668460 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.669300 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:09:59.669497 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:09:59.669606 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:09:59.672707 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:52699: BlockingRecv error: recv got EOF from 127.12.132.195:52699 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:00.415930 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:00.410880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:34837)
0620 08:10:00.411117 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:00.411131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:00.411193 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:00.411390 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:00.411391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:00.411574 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:00.411628 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.411630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.412426 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.412427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.412995 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.412996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.413388 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.413389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.413432 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.413878 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:00.413883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:00.415677 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":137,"threads_started":1}
W20260620 08:10:00.415993 22022 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:00.411060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:34837 (local address 127.12.132.254:38953)
0620 08:10:00.411192 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:00.411194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:00.411232 (+ 38us) server_negotiation.cc:407] Connection header received
0620 08:10:00.411262 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:00.411263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:00.411276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:00.411352 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:00.411793 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.412320 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.413110 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.413305 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.413366 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.415903 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:34837: BlockingRecv error: recv got EOF from 127.12.132.193:34837 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:00.559608 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:00.554427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:39543)
0620 08:10:00.554612 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:00.554630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:00.554697 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:00.554932 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:00.554934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:00.555163 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:00.555327 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.555330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.556052 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.556053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.556695 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.556696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.557062 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.557063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.557109 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.557559 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:00.557563 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:00.559549 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
W20260620 08:10:00.559746 22022 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:00.554568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:39543 (local address 127.12.132.254:38953)
0620 08:10:00.554641 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:00.554643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:00.554679 (+ 36us) server_negotiation.cc:407] Connection header received
0620 08:10:00.554769 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:00.554769 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:00.554780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:00.554846 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:00.555473 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.555963 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.556776 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.556956 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.557016 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.559632 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:39543: BlockingRecv error: recv got EOF from 127.12.132.194:39543 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:00 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:00.678135 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:00.673242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:60511)
0620 08:10:00.673397 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:00.673411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:00.673481 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:00.673671 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:00.673672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:00.673852 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:00.673898 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.673899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.674654 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.674655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.675419 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:00.675421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.675791 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.675792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:00.675834 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.676331 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:00.676336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:00.678076 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":54,"threads_started":1}
W20260620 08:10:00.678278 22022 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:00.673368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60511 (local address 127.12.132.254:38953)
0620 08:10:00.673454 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:00.673456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:00.673468 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:10:00.673541 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:00.673542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:00.673555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:00.673606 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:00.673974 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.674559 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.675513 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:00.675714 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:00.675777 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:00.678195 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:60511: BlockingRecv error: recv got EOF from 127.12.132.195:60511 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:01.422683 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:01.416880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:56913)
0620 08:10:01.417054 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:01.417066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:01.417174 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:01.417482 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:01.417484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:01.417739 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:01.417802 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.417805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.418554 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.418555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.419116 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.419118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.419473 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.419474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.419512 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.419979 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:01.419984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:01.422581 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
W20260620 08:10:01.422851 22026 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:01.417004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:56913 (local address 127.12.132.254:38953)
0620 08:10:01.417167 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:01.417169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:01.417180 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:10:01.417296 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:01.417297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:01.417311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:01.417394 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:01.417923 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.418463 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.419212 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.419403 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.419465 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.422748 (+ 3283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:56913: BlockingRecv error: recv got EOF from 127.12.132.193:56913 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:01.566042 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:01.560219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:53247)
0620 08:10:01.560520 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:01.560532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:01.560589 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:01.561008 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:01.561009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:01.561244 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:01.561308 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.561310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.562175 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.562176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.562787 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.562789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.563254 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.563255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.563317 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.563998 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:01.564009 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:01.565868 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":89,"threads_started":1}
W20260620 08:10:01.566147 22026 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:01.560468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:53247 (local address 127.12.132.254:38953)
0620 08:10:01.560766 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:01.560770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:01.560781 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:10:01.560819 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:01.560820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:01.560837 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:01.560956 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:01.561400 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.561996 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.562915 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.563152 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.563219 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.566046 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:53247: BlockingRecv error: recv got EOF from 127.12.132.194:53247 (error 108)
Metrics: {"server-negotiator.queue_time_us":256}
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:01 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:01.684158 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:01.678693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:58553)
0620 08:10:01.678828 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:01.678845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:01.678895 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:01.679048 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:01.679049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:01.679240 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:01.679286 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.679288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.680193 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.680194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.680781 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:01.680782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.681268 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.681268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:01.681306 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.681721 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:01.681725 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:01.684059 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260620 08:10:01.684208 22026 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:01.678779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:58553 (local address 127.12.132.254:38953)
0620 08:10:01.678867 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:01.678868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:01.678881 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:10:01.678955 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:01.678955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:01.678968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:01.679035 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:01.679368 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.680045 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.680896 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:01.681104 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:01.681264 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:01.684143 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:58553: BlockingRecv error: recv got EOF from 127.12.132.195:58553 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:02.429106 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:02.423285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:51139)
0620 08:10:02.423456 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:02.423467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:02.423519 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:02.423910 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:02.423912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:02.424142 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:02.424207 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.424210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.425137 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.425138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.425800 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.425803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.426274 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.426275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.426331 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.427148 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:02.427153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:02.429047 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
W20260620 08:10:02.429327 22030 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:02.423487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:51139 (local address 127.12.132.254:38953)
0620 08:10:02.423673 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:02.423675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:02.423686 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:10:02.423719 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:02.423720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:02.423734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:02.423837 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:02.424342 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.424996 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.425930 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.426160 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.426234 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.429226 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:51139: BlockingRecv error: recv got EOF from 127.12.132.193:51139 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:02.575398 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:02.566617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:36939)
0620 08:10:02.566810 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:02.566824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:02.566918 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:02.567079 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:02.567080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:02.567263 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:02.567313 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.567315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.568177 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.568182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.569286 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.569289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.569654 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.569655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.569697 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.573189 (+ 3492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:02.573195 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:02.575299 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":113,"threads_started":1}
W20260620 08:10:02.575452 22030 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:02.566708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:36939 (local address 127.12.132.254:38953)
0620 08:10:02.566817 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:02.566819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:02.566898 (+ 79us) server_negotiation.cc:407] Connection header received
0620 08:10:02.566988 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:02.566989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:02.567001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:02.567074 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:02.567390 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.568009 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.569408 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.569590 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.569646 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.575389 (+ 5743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:36939: BlockingRecv error: recv got EOF from 127.12.132.194:36939 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:02 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.195@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:02.690070 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:02.684717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.195:45939)
0620 08:10:02.684908 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:02.684919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:02.685041 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:02.685179 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:02.685180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:02.685345 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:02.685390 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.685391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.686088 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.686089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.686641 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:02.686643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.687007 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.687008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:02.687061 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.687685 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:02.687693 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:02.689953 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":108,"threads_started":1}
W20260620 08:10:02.690104 22030 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:02.684786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:45939 (local address 127.12.132.254:38953)
0620 08:10:02.684865 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:02.684868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:02.685018 (+ 150us) server_negotiation.cc:407] Connection header received
0620 08:10:02.685098 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:02.685098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:02.685111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:02.685161 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:02.685455 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.685966 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.686733 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:02.686926 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:02.687008 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:02.690035 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.195:45939: BlockingRecv error: recv got EOF from 127.12.132.195:45939 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 08:10:03 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:03 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.193@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:03.434679 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:03.429826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.193:57815)
0620 08:10:03.430004 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:03.430017 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:03.430185 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:03.430382 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:03.430383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:03.430560 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:03.430610 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.430612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.431302 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.431303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.431869 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.431870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.432212 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.432213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.432252 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.432777 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:03.432781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:03.434620 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260620 08:10:03.434886 22034 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:03.429903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:57815 (local address 127.12.132.254:38953)
0620 08:10:03.430166 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:03.430168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:03.430190 (+ 22us) server_negotiation.cc:407] Connection header received
0620 08:10:03.430254 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:03.430255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:03.430266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:03.430337 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:03.430692 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.431215 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.431952 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.432144 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.432199 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.434784 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.193:57815: BlockingRecv error: recv got EOF from 127.12.132.193:57815 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":196,"threads_started":1}
Jun 20 08:10:03 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
Jun 20 08:10:03 dist-test-slave-z0sp krb5kdc[20976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.132.194@KRBTEST.COM for oryx/127.12.132.254@KRBTEST.COM, Server not found in Kerberos database
I20260620 08:10:03.581568 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:03.575978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38953 (local address 127.12.132.194:58075)
0620 08:10:03.576226 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:03.576246 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:03.576307 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:03.576516 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:03.576518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:03.576675 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:03.576723 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.576724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.577632 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.577634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.578267 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.578271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.578748 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.578749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.578819 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.579458 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:03.579462 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:03.581502 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.132.254:38953: Server oryx/127.12.132.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":162,"threads_started":1}
W20260620 08:10:03.581712 22034 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:03.576185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:58075 (local address 127.12.132.254:38953)
0620 08:10:03.576276 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:03.576279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:03.576294 (+ 15us) server_negotiation.cc:407] Connection header received
0620 08:10:03.576378 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:03.576379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:03.576391 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:03.576444 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:03.576886 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.577470 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.578392 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.578662 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.578732 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.581630 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.132.194:58075: BlockingRecv error: recv got EOF from 127.12.132.194:58075 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 08:10:03.601966 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21560
I20260620 08:10:03.607244 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21711
I20260620 08:10:03.611438 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21844
I20260620 08:10:03.615459 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 21489
2026-06-20T08:10:03Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T08:10:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:10:03Z Disabled control of system clock
I20260620 08:10:03.644748 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:42407
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:42407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 08:10:03.729089 22039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:03.729357 22039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:03.729379 22039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:03.731272 22039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:10:03.731348 22039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:03.731362 22039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:03.731376 22039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:10:03.731390 22039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:10:03.733219 22039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:42407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:42407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:03.733419 22039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:03.733660 22039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:03.739159 22044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:03.739154 22047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:03.739354 22045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:03.739508 22039 server_base.cc:1034] running on GCE node
I20260620 08:10:03.739764 22039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:03.740101 22039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:03.741293 22039 hybrid_clock.cc:648] HybridClock initialized: now 1781943003741278 us; error 29 us; skew 500 ppm
I20260620 08:10:03.742673 22039 webserver.cc:460] Webserver started at http://127.12.132.254:38037/ using document root <none> and password file <none>
I20260620 08:10:03.742940 22039 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:03.742987 22039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:03.743145 22039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:03.744230 22039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "dd7a9651da6842adaedc633ca9d592b1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "859797021d3d4b6b5ec1ba3d7150e80c"
server_key_iv: "2d8b4e9f5d134a3f0f3c9e99a9e7afae"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.744618 22039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "dd7a9651da6842adaedc633ca9d592b1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "859797021d3d4b6b5ec1ba3d7150e80c"
server_key_iv: "2d8b4e9f5d134a3f0f3c9e99a9e7afae"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.745973 22039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:03.747128 22053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:03.747605 22039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:03.747771 22039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "dd7a9651da6842adaedc633ca9d592b1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "859797021d3d4b6b5ec1ba3d7150e80c"
server_key_iv: "2d8b4e9f5d134a3f0f3c9e99a9e7afae"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.748031 22039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:03.762336 22039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:03.762669 22039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:03.762782 22039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:03.766782 22039 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:42407
I20260620 08:10:03.766857 22105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:42407 every 8 connection(s)
I20260620 08:10:03.767185 22039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:10:03.767884 22106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:03.770308 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22039
I20260620 08:10:03.770411 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:10:03.770619 12819 external_mini_cluster.cc:1428] Setting key afbdbd283717614174eb90175b7ac226
I20260620 08:10:03.770627 22106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1: Bootstrap starting.
I20260620 08:10:03.771279 22106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:03.771625 22106 log.cc:824] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:03.772385 22106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1: No bootstrap required, opened a new log
I20260620 08:10:03.773902 22106 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } }
I20260620 08:10:03.774330 22106 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:03.774433 22106 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7a9651da6842adaedc633ca9d592b1, State: Initialized, Role: FOLLOWER
I20260620 08:10:03.774657 22106 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } }
I20260620 08:10:03.774814 22106 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:10:03.774878 22106 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:10:03.774943 22106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:03.776471 22106 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } }
I20260620 08:10:03.776746 22106 leader_election.cc:302] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd7a9651da6842adaedc633ca9d592b1; no voters:
I20260620 08:10:03.776945 22106 leader_election.cc:288] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:10:03.776988 22111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:03.777055 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:03.771416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.12.132.254:42407)
0620 08:10:03.771611 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:03.771618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:03.771642 (+ 24us) server_negotiation.cc:407] Connection header received
0620 08:10:03.772107 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:03.772113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:03.772167 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:03.772279 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:03.772543 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.773435 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.774719 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.775264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.775367 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.776230 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:03.776233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:03.776233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:03.776401 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:03.776433 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:03.776443 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:03.776518 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:03.776750 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 08:10:03.776833 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260620 08:10:03.777109 22111 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [term 1 LEADER]: Becoming Leader. State: Replica: dd7a9651da6842adaedc633ca9d592b1, State: Running, Role: LEADER
I20260620 08:10:03.777194 22111 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } }
I20260620 08:10:03.777230 22106 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:10:03.777491 22111 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd7a9651da6842adaedc633ca9d592b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } } }
I20260620 08:10:03.777556 22111 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:03.777612 22112 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd7a9651da6842adaedc633ca9d592b1. Latest consensus state: current_term: 1 leader_uuid: "dd7a9651da6842adaedc633ca9d592b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd7a9651da6842adaedc633ca9d592b1" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42407 } } }
I20260620 08:10:03.778208 22112 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:03.778378 22115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:10:03.778762 22115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:10:03.780467 22115 catalog_manager.cc:1347] Generated new cluster ID: 2d36dcda85f845789bfccf7f2d1cbd31
I20260620 08:10:03.780575 22115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:10:03.785983 22115 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:10:03.786504 22115 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:10:03.792048 22115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd7a9651da6842adaedc633ca9d592b1: Generated new TSK 0
I20260620 08:10:03.792209 22115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:10:03.797535 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--builtin_ntp_servers=127.12.132.212:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 08:10:03.877969 22130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:03.878202 22130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:03.878225 22130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:03.879423 22130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:03.879470 22130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:03.879498 22130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:10:03.880831 22130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:03.881058 22130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:03.881353 22130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:03.887193 22135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:03.887218 22136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:03.887223 22138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:03.887674 22130 server_base.cc:1034] running on GCE node
I20260620 08:10:03.887878 22130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:03.888134 22130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:03.889315 22130 hybrid_clock.cc:648] HybridClock initialized: now 1781943003889295 us; error 32 us; skew 500 ppm
I20260620 08:10:03.890648 22130 webserver.cc:460] Webserver started at http://127.12.132.193:36321/ using document root <none> and password file <none>
I20260620 08:10:03.890933 22130 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:03.890981 22130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:03.891113 22130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:03.892158 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "a5a1d5cfb1c54ee6993f14622654e4f1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "56b9380027a62c57657a07378fe66d8f"
server_key_iv: "4a02a1deb12f32e5fe9f4ec717f511c5"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.892571 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "a5a1d5cfb1c54ee6993f14622654e4f1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "56b9380027a62c57657a07378fe66d8f"
server_key_iv: "4a02a1deb12f32e5fe9f4ec717f511c5"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.893961 22130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:03.894727 22144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:03.894901 22130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:10:03.894985 22130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "a5a1d5cfb1c54ee6993f14622654e4f1"
format_stamp: "Formatted at 2026-06-20 08:10:03 on dist-test-slave-z0sp"
server_key: "56b9380027a62c57657a07378fe66d8f"
server_key_iv: "4a02a1deb12f32e5fe9f4ec717f511c5"
server_key_version: "encryptionkey@0"
I20260620 08:10:03.895058 22130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:03.918457 22130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:03.918781 22130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:03.918926 22130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:03.919193 22130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:03.919545 22130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:03.919603 22130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:03.919638 22130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:03.919667 22130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:03.925650 22130 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:38807
I20260620 08:10:03.925693 22257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:38807 every 8 connection(s)
I20260620 08:10:03.926039 22130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:10:03.932129 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:03.926778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:34779 (local address 127.12.132.254:42407)
0620 08:10:03.926854 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:03.926856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:03.927106 (+ 250us) server_negotiation.cc:407] Connection header received
0620 08:10:03.927641 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:03.927642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:03.927653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:03.927739 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:03.928939 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.929469 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.930560 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.930750 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.930828 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.931832 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:03.931833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:03.931833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:03.931837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:03.931851 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:03.931855 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:03.931907 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:03.932046 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 08:10:03.932068 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 08:10:03.932422 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:03.926755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42407 (local address 127.12.132.193:34779)
0620 08:10:03.926969 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:03.927011 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:03.927524 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:03.927967 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:03.927973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:03.927987 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:03.928790 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.928799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.929620 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.929622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.930455 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:03.930458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:03.930894 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:03.930895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:03.931018 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:03.931630 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:03.931630 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:03.931671 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:03.931705 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:03.931950 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:03.931956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:03.931959 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:03.932037 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 08:10:03.932128 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260620 08:10:03.933059 22258 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42407
I20260620 08:10:03.933141 22258 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:03.933157 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22130
I20260620 08:10:03.933256 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:10:03.933359 22258 heartbeater.cc:511] Master 127.12.132.254:42407 requested a full tablet report, sending...
I20260620 08:10:03.933488 12819 external_mini_cluster.cc:1428] Setting key 7c93122a0d8c067d4f502d1da5cc47a5
I20260620 08:10:03.933926 22070 ts_manager.cc:194] Registered new tserver with Master: a5a1d5cfb1c54ee6993f14622654e4f1 (127.12.132.193:38807)
I20260620 08:10:03.934540 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--builtin_ntp_servers=127.12.132.212:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 08:10:03.934852 22070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.193:34779
W20260620 08:10:04.018525 22261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.018709 22261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.018729 22261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.019866 22261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.019906 22261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.019932 22261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:10:04.021199 22261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.021376 22261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.021608 22261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.026425 22269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.026506 22266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.026506 22267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.026724 22261 server_base.cc:1034] running on GCE node
I20260620 08:10:04.026932 22261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.027175 22261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.028344 22261 hybrid_clock.cc:648] HybridClock initialized: now 1781943004028324 us; error 41 us; skew 500 ppm
I20260620 08:10:04.029564 22261 webserver.cc:460] Webserver started at http://127.12.132.194:36365/ using document root <none> and password file <none>
I20260620 08:10:04.029808 22261 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.029873 22261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.030011 22261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.031040 22261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "77ac748b2432462ea561e891eaf75819"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "cdb2a5f275d00b6dc97d36c7393ea99b"
server_key_iv: "1d1d2767b2204be13f7e95d9065e053f"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.031380 22261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "77ac748b2432462ea561e891eaf75819"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "cdb2a5f275d00b6dc97d36c7393ea99b"
server_key_iv: "1d1d2767b2204be13f7e95d9065e053f"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.032763 22261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:10:04.033524 22275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.033694 22261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.033803 22261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "77ac748b2432462ea561e891eaf75819"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "cdb2a5f275d00b6dc97d36c7393ea99b"
server_key_iv: "1d1d2767b2204be13f7e95d9065e053f"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.033890 22261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.048151 22261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.048435 22261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.048632 22261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.048852 22261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:04.049176 22261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:04.049228 22261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.049268 22261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:04.049296 22261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.055523 22261 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:39215
I20260620 08:10:04.055572 22388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:39215 every 8 connection(s)
I20260620 08:10:04.055876 22261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:10:04.059755 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22261
I20260620 08:10:04.059878 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:10:04.060092 12819 external_mini_cluster.cc:1428] Setting key e7988fd85ffa2147e3571ced131483b1
I20260620 08:10:04.061223 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.056578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:32839 (local address 127.12.132.254:42407)
0620 08:10:04.056653 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.056656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.056909 (+ 253us) server_negotiation.cc:407] Connection header received
0620 08:10:04.057384 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.057385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.057395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.057457 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.058110 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.058611 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.059573 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.059757 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.059858 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:04.060951 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.060953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.060953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.060957 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.060975 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.060979 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.061046 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.061098 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.061123 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 08:10:04.061232 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--builtin_ntp_servers=127.12.132.212:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 08:10:04.061372 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.056560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42407 (local address 127.12.132.194:32839)
0620 08:10:04.056783 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:04.056826 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:04.057300 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:04.057509 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:04.057512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:04.057520 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:04.058021 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:04.058026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.058687 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.058688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:04.059445 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:04.059447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.059850 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.059852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:04.060020 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:04.060790 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:04.060791 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:04.060831 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:04.060858 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:04.061046 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:04.061049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:04.061051 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:10:04.061096 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 08:10:04.061167 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260620 08:10:04.061811 22389 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42407
I20260620 08:10:04.061913 22389 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:04.062083 22389 heartbeater.cc:511] Master 127.12.132.254:42407 requested a full tablet report, sending...
I20260620 08:10:04.062587 22070 ts_manager.cc:194] Registered new tserver with Master: 77ac748b2432462ea561e891eaf75819 (127.12.132.194:39215)
I20260620 08:10:04.063022 22070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.194:32839
W20260620 08:10:04.147426 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.147662 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.147694 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.148837 22392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.148900 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.148944 22392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:10:04.150255 22392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.150516 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.150871 22392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.156100 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.156145 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.156091 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.156366 22392 server_base.cc:1034] running on GCE node
I20260620 08:10:04.156603 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.156894 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.158075 22392 hybrid_clock.cc:648] HybridClock initialized: now 1781943004158065 us; error 40 us; skew 500 ppm
I20260620 08:10:04.159369 22392 webserver.cc:460] Webserver started at http://127.12.132.195:46283/ using document root <none> and password file <none>
I20260620 08:10:04.159623 22392 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.159696 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.159833 22392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.160960 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "40aaf7a2ae41482db9fe0e4e1412a090"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "a7bf38c33ebfe734ee9893f8842d789a"
server_key_iv: "3c32eab9fde6170c849bb229c3890339"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.161316 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "40aaf7a2ae41482db9fe0e4e1412a090"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "a7bf38c33ebfe734ee9893f8842d789a"
server_key_iv: "3c32eab9fde6170c849bb229c3890339"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.162886 22392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:10:04.164114 22406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.164392 22392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:04.164530 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "40aaf7a2ae41482db9fe0e4e1412a090"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "a7bf38c33ebfe734ee9893f8842d789a"
server_key_iv: "3c32eab9fde6170c849bb229c3890339"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.164625 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.192795 22392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.193123 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.193279 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.193508 22392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:04.193838 22392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:04.193899 22392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.193940 22392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:04.193967 22392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.200011 22392 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:38137
I20260620 08:10:04.200139 22519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:38137 every 8 connection(s)
I20260620 08:10:04.200471 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:10:04.205554 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.201280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:54667 (local address 127.12.132.254:42407)
0620 08:10:04.201331 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.201333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.201564 (+ 231us) server_negotiation.cc:407] Connection header received
0620 08:10:04.202027 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.202028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.202038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.202103 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.202699 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.203180 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.204177 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.204367 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.204427 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:04.205319 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.205320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.205320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.205325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.205338 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.205341 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.205393 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.205475 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.205496 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 08:10:04.205785 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.201227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42407 (local address 127.12.132.195:54667)
0620 08:10:04.201413 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:04.201453 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:04.201919 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:04.202152 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:04.202155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:04.202163 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:04.202625 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:04.202629 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.203266 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.203267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:04.204080 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:04.204083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:04.204466 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:04.204467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:04.204594 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:04.205173 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:04.205174 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:04.205200 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:04.205222 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:04.205403 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:04.205406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:04.205408 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:10:04.205463 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 08:10:04.205538 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260620 08:10:04.206480 22520 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42407
I20260620 08:10:04.206643 22520 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:04.206915 22520 heartbeater.cc:511] Master 127.12.132.254:42407 requested a full tablet report, sending...
I20260620 08:10:04.207422 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22392
I20260620 08:10:04.207530 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:10:04.207764 12819 external_mini_cluster.cc:1428] Setting key 8d9512e91495cd1ec4b2b9d2ae0752b0
I20260620 08:10:04.207741 22070 ts_manager.cc:194] Registered new tserver with Master: 40aaf7a2ae41482db9fe0e4e1412a090 (127.12.132.195:38137)
I20260620 08:10:04.208415 22070 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.195:54667
I20260620 08:10:04.209195 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 08:10:04.211066 22109 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:04.210537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.12.132.254:42407)
0620 08:10:04.210576 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.210578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.210727 (+ 149us) server_negotiation.cc:407] Connection header received
0620 08:10:04.210821 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.210822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.210832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.210893 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.211004 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53090: BlockingRecv error: recv got EOF from 127.0.0.1:53090 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:10:04.211364 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22130
I20260620 08:10:04.216557 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22261
I20260620 08:10:04.220890 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22392
I20260620 08:10:04.225026 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22039
2026-06-20T08:10:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (608 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T08:10:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:10:04Z Disabled control of system clock
I20260620 08:10:04.251860 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:40441
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:40441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 08:10:04.332418 22531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.332644 22531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.332665 22531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.333841 22531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:10:04.333885 22531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.333899 22531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.333910 22531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:10:04.333923 22531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:10:04.335321 22531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:40441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.335562 22531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.335865 22531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.340730 22536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.340730 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.340874 22537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.341111 22531 server_base.cc:1034] running on GCE node
I20260620 08:10:04.341506 22531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.341923 22531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.343155 22531 hybrid_clock.cc:648] HybridClock initialized: now 1781943004343126 us; error 43 us; skew 500 ppm
I20260620 08:10:04.344700 22531 webserver.cc:460] Webserver started at http://127.12.132.254:36595/ using document root <none> and password file <none>
I20260620 08:10:04.344997 22531 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.345064 22531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.345206 22531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.346292 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "c16ecb51bca84d31a23e9165be7072e4"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "62e269505a170407052ca7bb43e9e88c"
server_key_iv: "57259d9ee5d0ae3d92d7f2e900838f00"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.346587 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "c16ecb51bca84d31a23e9165be7072e4"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "62e269505a170407052ca7bb43e9e88c"
server_key_iv: "57259d9ee5d0ae3d92d7f2e900838f00"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.347781 22531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:10:04.348531 22545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.348688 22531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:04.348763 22531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "c16ecb51bca84d31a23e9165be7072e4"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "62e269505a170407052ca7bb43e9e88c"
server_key_iv: "57259d9ee5d0ae3d92d7f2e900838f00"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.348816 22531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.351956 22531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.352162 22531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.352252 22531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.356066 22531 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:40441
I20260620 08:10:04.356137 22597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:40441 every 8 connection(s)
I20260620 08:10:04.356447 22531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:10:04.357122 22598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:04.359782 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4: Bootstrap starting.
I20260620 08:10:04.360366 22598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:04.360702 22598 log.cc:824] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:04.361352 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4: No bootstrap required, opened a new log
I20260620 08:10:04.362955 22598 raft_consensus.cc:348] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } }
I20260620 08:10:04.363086 22598 raft_consensus.cc:374] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:04.363109 22598 raft_consensus.cc:729] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c16ecb51bca84d31a23e9165be7072e4, State: Initialized, Role: FOLLOWER
I20260620 08:10:04.363252 22598 consensus_queue.cc:260] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } }
I20260620 08:10:04.363328 22598 raft_consensus.cc:388] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:10:04.363351 22598 raft_consensus.cc:482] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:10:04.363432 22598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:04.363970 22598 raft_consensus.cc:504] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } }
I20260620 08:10:04.364051 22598 leader_election.cc:302] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c16ecb51bca84d31a23e9165be7072e4; no voters:
I20260620 08:10:04.364234 22598 leader_election.cc:288] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:10:04.364284 22601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:04.364470 22601 raft_consensus.cc:686] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [term 1 LEADER]: Becoming Leader. State: Replica: c16ecb51bca84d31a23e9165be7072e4, State: Running, Role: LEADER
I20260620 08:10:04.364583 22601 consensus_queue.cc:237] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } }
I20260620 08:10:04.364789 22598 sys_catalog.cc:564] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:10:04.364892 22603 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c16ecb51bca84d31a23e9165be7072e4. Latest consensus state: current_term: 1 leader_uuid: "c16ecb51bca84d31a23e9165be7072e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } } }
I20260620 08:10:04.364998 22603 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:04.365137 22602 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c16ecb51bca84d31a23e9165be7072e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ecb51bca84d31a23e9165be7072e4" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40441 } } }
I20260620 08:10:04.365216 22602 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:04.365262 22605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:10:04.365758 22605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:10:04.366393 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22531
I20260620 08:10:04.366497 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:10:04.366721 12819 external_mini_cluster.cc:1428] Setting key 48c8437a703d2e2d2f068d9169c3c2a6
I20260620 08:10:04.367295 22605 catalog_manager.cc:1347] Generated new cluster ID: 168ec99d7ab44a318dfe7d85d820ad14
I20260620 08:10:04.367372 22605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:10:04.368968 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.367166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33666 (local address 127.12.132.254:40441)
0620 08:10:04.367376 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.367384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.367574 (+ 190us) server_negotiation.cc:407] Connection header received
0620 08:10:04.367709 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.367716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.367759 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.367888 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.368548 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.368551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.368551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.368554 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.368592 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.368606 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.368665 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.368754 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.368811 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260620 08:10:04.406916 22605 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:10:04.407171 22605 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:10:04.410708 22605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c16ecb51bca84d31a23e9165be7072e4: Generated new TSK 0
I20260620 08:10:04.410909 22605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:10:04.417644 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--builtin_ntp_servers=127.12.132.212:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 08:10:04.533164 22622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.533370 22622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.533394 22622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.534477 22622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.534515 22622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.534545 22622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:10:04.535797 22622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.535972 22622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.536202 22622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.541294 22628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.541355 22627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.541448 22622 server_base.cc:1034] running on GCE node
W20260620 08:10:04.541355 22630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.541859 22622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.542152 22622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.543311 22622 hybrid_clock.cc:648] HybridClock initialized: now 1781943004543302 us; error 52 us; skew 500 ppm
I20260620 08:10:04.544783 22622 webserver.cc:460] Webserver started at http://127.12.132.193:34483/ using document root <none> and password file <none>
I20260620 08:10:04.545064 22622 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.545137 22622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.545289 22622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.546402 22622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "f319fe4d5c97454dbe7cecfc0172bdb6"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "2bdee370c2f429920b62a8b4103c22fe"
server_key_iv: "c50ef255eda076127e5bfeeed33bfa3b"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.546762 22622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "f319fe4d5c97454dbe7cecfc0172bdb6"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "2bdee370c2f429920b62a8b4103c22fe"
server_key_iv: "c50ef255eda076127e5bfeeed33bfa3b"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.548265 22622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:10:04.549294 22636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.549641 22622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:10:04.549791 22622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "f319fe4d5c97454dbe7cecfc0172bdb6"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "2bdee370c2f429920b62a8b4103c22fe"
server_key_iv: "c50ef255eda076127e5bfeeed33bfa3b"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.549929 22622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.553819 22622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.554085 22622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.554235 22622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.554450 22622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:04.554744 22622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:04.554795 22622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.554844 22622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:04.554880 22622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.560688 22622 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:34689
I20260620 08:10:04.560745 22749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:34689 every 8 connection(s)
I20260620 08:10:04.561054 22622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:10:04.563683 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22622
I20260620 08:10:04.563793 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:10:04.564007 12819 external_mini_cluster.cc:1428] Setting key 01f4c95ae8de03b82148829e3a1608d4
I20260620 08:10:04.564445 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.561791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52671 (local address 127.12.132.254:40441)
0620 08:10:04.561856 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.561858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.562091 (+ 233us) server_negotiation.cc:407] Connection header received
0620 08:10:04.562585 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.562586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.562593 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.562652 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.563955 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.563959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.563959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.563967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.564001 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.564008 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.564116 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.564351 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.564375 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 08:10:04.564805 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.561785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:40441 (local address 127.12.132.193:52671)
0620 08:10:04.561963 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:04.561993 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:04.562480 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:04.562717 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:04.562723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:04.562735 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:04.563703 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:04.563704 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:04.563746 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:04.563787 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:04.564168 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:04.564204 (+ 36us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:04.564210 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 08:10:04.564324 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 08:10:04.564433 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260620 08:10:04.565209 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--builtin_ntp_servers=127.12.132.212:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 08:10:04.565513 22750 heartbeater.cc:347] Connected to a master server at 127.12.132.254:40441
I20260620 08:10:04.565618 22750 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:04.565769 22750 heartbeater.cc:511] Master 127.12.132.254:40441 requested a full tablet report, sending...
I20260620 08:10:04.566375 22562 ts_manager.cc:194] Registered new tserver with Master: f319fe4d5c97454dbe7cecfc0172bdb6 (127.12.132.193:34689)
W20260620 08:10:04.646173 22753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.646598 22753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.646636 22753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.648763 22753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.648844 22753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.648887 22753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:10:04.650924 22753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.651172 22753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.651458 22753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.656643 22759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.656667 22761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.656785 22753 server_base.cc:1034] running on GCE node
W20260620 08:10:04.656883 22758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.657099 22753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.657332 22753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.658499 22753 hybrid_clock.cc:648] HybridClock initialized: now 1781943004658485 us; error 28 us; skew 500 ppm
I20260620 08:10:04.659797 22753 webserver.cc:460] Webserver started at http://127.12.132.194:39333/ using document root <none> and password file <none>
I20260620 08:10:04.660086 22753 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.660168 22753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.660317 22753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.661427 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "09b0e04d27074b1a94266945d48f4260"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "3fc666016b45640f481a266eb8bcc2af"
server_key_iv: "827cb1d8997364e32831f633461f2b06"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.661798 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "09b0e04d27074b1a94266945d48f4260"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "3fc666016b45640f481a266eb8bcc2af"
server_key_iv: "827cb1d8997364e32831f633461f2b06"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.663301 22753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:10:04.664127 22767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.664403 22753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:04.664543 22753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "09b0e04d27074b1a94266945d48f4260"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "3fc666016b45640f481a266eb8bcc2af"
server_key_iv: "827cb1d8997364e32831f633461f2b06"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.664633 22753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.668006 22753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.668258 22753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.668403 22753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.668630 22753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:04.668937 22753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:04.668998 22753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.669037 22753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:04.669066 22753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.674778 22753 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:33675
I20260620 08:10:04.674818 22880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:33675 every 8 connection(s)
I20260620 08:10:04.675138 22753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:10:04.678320 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.675940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:58531 (local address 127.12.132.254:40441)
0620 08:10:04.676139 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.676141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.676312 (+ 171us) server_negotiation.cc:407] Connection header received
0620 08:10:04.676864 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.676865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.676872 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.676923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.678083 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.678084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.678084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.678089 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.678105 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.678110 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.678162 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.678235 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.678249 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 08:10:04.678503 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.675808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:40441 (local address 127.12.132.194:58531)
0620 08:10:04.676180 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:04.676225 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:04.676753 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:04.676983 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:04.676986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:04.676997 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:04.677939 (+ 942us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:04.677940 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:04.677965 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:04.677990 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:04.678158 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:04.678161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:04.678164 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:04.678238 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 08:10:04.678302 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
I20260620 08:10:04.679003 22881 heartbeater.cc:347] Connected to a master server at 127.12.132.254:40441
I20260620 08:10:04.679082 22881 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:04.679247 22881 heartbeater.cc:511] Master 127.12.132.254:40441 requested a full tablet report, sending...
I20260620 08:10:04.679669 22562 ts_manager.cc:194] Registered new tserver with Master: 09b0e04d27074b1a94266945d48f4260 (127.12.132.194:33675)
I20260620 08:10:04.680470 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22753
I20260620 08:10:04.680648 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:10:04.680806 12819 external_mini_cluster.cc:1428] Setting key 15ec4c2b416f4e2562300c449296e885
I20260620 08:10:04.682025 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--builtin_ntp_servers=127.12.132.212:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 08:10:04.765084 22884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:04.765337 22884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:04.765384 22884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:04.766705 22884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:04.766745 22884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:04.766799 22884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:10:04.768216 22884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:40441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:04.768464 22884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:04.768755 22884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:04.774055 22890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.774132 22892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:04.774158 22889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:04.774228 22884 server_base.cc:1034] running on GCE node
I20260620 08:10:04.774451 22884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:04.774711 22884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:04.775882 22884 hybrid_clock.cc:648] HybridClock initialized: now 1781943004775859 us; error 36 us; skew 500 ppm
I20260620 08:10:04.777110 22884 webserver.cc:460] Webserver started at http://127.12.132.195:42179/ using document root <none> and password file <none>
I20260620 08:10:04.777355 22884 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:04.777426 22884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:04.777561 22884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:04.778591 22884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "f113e1f28d3243dcb2860585a602b990"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "e6ab673c43ef075c018d93adfd4af381"
server_key_iv: "e047cb8cbd4609b931744a5118ddce0e"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.778956 22884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "f113e1f28d3243dcb2860585a602b990"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "e6ab673c43ef075c018d93adfd4af381"
server_key_iv: "e047cb8cbd4609b931744a5118ddce0e"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.780321 22884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:10:04.781206 22898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.781388 22884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:04.781486 22884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "f113e1f28d3243dcb2860585a602b990"
format_stamp: "Formatted at 2026-06-20 08:10:04 on dist-test-slave-z0sp"
server_key: "e6ab673c43ef075c018d93adfd4af381"
server_key_iv: "e047cb8cbd4609b931744a5118ddce0e"
server_key_version: "encryptionkey@0"
I20260620 08:10:04.781586 22884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:04.785655 22884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:04.786067 22884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:04.786338 22884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:04.786634 22884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:04.786995 22884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:04.787037 22884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.787058 22884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:04.787071 22884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:04.793375 22884 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:43237
I20260620 08:10:04.793453 23011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:43237 every 8 connection(s)
I20260620 08:10:04.793782 22884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:10:04.796978 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 22884
I20260620 08:10:04.797114 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:10:04.797153 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.794603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:41011 (local address 127.12.132.254:40441)
0620 08:10:04.794686 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.794689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.794948 (+ 259us) server_negotiation.cc:407] Connection header received
0620 08:10:04.795338 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.795339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.795344 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.795406 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.796888 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:04.796889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:04.796889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:10:04.796907 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:04.796920 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:10:04.796924 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:04.796993 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:04.797066 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 08:10:04.797081 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 08:10:04.797346 12819 external_mini_cluster.cc:1428] Setting key cc814d1669c52d762ba7b987d760d9ab
I20260620 08:10:04.797354 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:04.794530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:40441 (local address 127.12.132.195:41011)
0620 08:10:04.794735 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:04.794771 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:04.795228 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:04.795467 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:04.795470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:04.795477 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:04.796728 (+ 1251us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:10:04.796728 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:04.796756 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:10:04.796783 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:04.796991 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:04.796996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:04.796999 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:04.797077 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 08:10:04.797137 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260620 08:10:04.797760 23012 heartbeater.cc:347] Connected to a master server at 127.12.132.254:40441
I20260620 08:10:04.797868 23012 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:04.798022 23012 heartbeater.cc:511] Master 127.12.132.254:40441 requested a full tablet report, sending...
I20260620 08:10:04.798372 22561 ts_manager.cc:194] Registered new tserver with Master: f113e1f28d3243dcb2860585a602b990 (127.12.132.195:43237)
I20260620 08:10:04.809414 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 08:10:04.812091 22621 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:10:04.811269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.12.132.254:40441)
0620 08:10:04.811400 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:04.811404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:04.811536 (+ 132us) server_negotiation.cc:407] Connection header received
0620 08:10:04.811780 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:04.811781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:04.811788 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:04.811868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:04.812013 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51616: BlockingRecv error: recv got EOF from 127.0.0.1:51616 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 08:10:04.812222 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22622
I20260620 08:10:04.817101 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22753
I20260620 08:10:04.821692 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22884
I20260620 08:10:04.825793 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 22531
2026-06-20T08:10:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (603 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:10:04 dist-test-slave-z0sp krb5kdc[23022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:10:04 dist-test-slave-z0sp krb5kdc[23022](info): set up 2 sockets
Jun 20 08:10:04 dist-test-slave-z0sp krb5kdc[23022](info): commencing operation
krb5kdc: starting...
W20260620 08:10:05.886164 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:10:05 dist-test-slave-z0sp krb5kdc[23022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:10:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:10:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:10:06.012149 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38409
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:06.098553 23035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:06.098805 23035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:06.098845 23035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:06.100293 23035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:10:06.100564 23035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:06.100643 23035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:06.100680 23035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:10:06.100711 23035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:10:06.102821 23035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:06.103127 23035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:06.103405 23035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:06.110252 23043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.110415 23040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.110611 23035 server_base.cc:1034] running on GCE node
W20260620 08:10:06.110258 23041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.111033 23035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:06.111311 23035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:06.112624 23035 hybrid_clock.cc:648] HybridClock initialized: now 1781943006112600 us; error 45 us; skew 500 ppm
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:06.116197 23035 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:10:06.116746 23035 webserver.cc:460] Webserver started at http://127.12.132.254:36441/ using document root <none> and password file <none>
I20260620 08:10:06.116979 23035 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:06.117046 23035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:06.117197 23035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:06.118778 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ba13f9d1a67d4db4afa0c2670e3e4212"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "52e87eaada62c53eb52d807932d48834"
server_key_iv: "6aae41ab3be068f560cb224035c60e4d"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.119266 23035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ba13f9d1a67d4db4afa0c2670e3e4212"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "52e87eaada62c53eb52d807932d48834"
server_key_iv: "6aae41ab3be068f560cb224035c60e4d"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.121093 23035 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 08:10:06.122088 23050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.122328 23035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:06.122437 23035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ba13f9d1a67d4db4afa0c2670e3e4212"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "52e87eaada62c53eb52d807932d48834"
server_key_iv: "6aae41ab3be068f560cb224035c60e4d"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.122501 23035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:06.143296 23035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:06.145418 23035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:06.145531 23035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:06.150195 23035 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38409
I20260620 08:10:06.150230 23102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38409 every 8 connection(s)
I20260620 08:10:06.150578 23035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:10:06.151594 23103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:06.153851 23103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: Bootstrap starting.
I20260620 08:10:06.154444 23103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:06.154737 23103 log.cc:824] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:06.155438 23103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: No bootstrap required, opened a new log
I20260620 08:10:06.157101 23103 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } }
I20260620 08:10:06.157263 23103 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:06.157310 23103 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba13f9d1a67d4db4afa0c2670e3e4212, State: Initialized, Role: FOLLOWER
I20260620 08:10:06.157455 23103 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } }
I20260620 08:10:06.157538 23103 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:10:06.157572 23103 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:10:06.157625 23103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:06.158280 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 23035
I20260620 08:10:06.158425 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:10:06.158619 12819 external_mini_cluster.cc:1428] Setting key 78c25480f048ef149f07aa5318fea21e
I20260620 08:10:06.158668 23103 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } }
I20260620 08:10:06.158790 23103 leader_election.cc:302] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba13f9d1a67d4db4afa0c2670e3e4212; no voters:
I20260620 08:10:06.158951 23103 leader_election.cc:288] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:10:06.159024 23106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:06.159214 23103 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:10:06.159235 23106 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [term 1 LEADER]: Becoming Leader. State: Replica: ba13f9d1a67d4db4afa0c2670e3e4212, State: Running, Role: LEADER
I20260620 08:10:06.159413 23106 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } }
I20260620 08:10:06.161072 23107 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba13f9d1a67d4db4afa0c2670e3e4212. Latest consensus state: current_term: 1 leader_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } } }
I20260620 08:10:06.161063 23109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba13f9d1a67d4db4afa0c2670e3e4212" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38409 } } }
I20260620 08:10:06.161484 23109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [sys.catalog]: This master's current role is: LEADER
W20260620 08:10:06.161734 23123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 08:10:06.161774 23107 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212 [sys.catalog]: This master's current role is: LEADER
W20260620 08:10:06.161859 23123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:10:06.162101 23125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:10:06.162266 23125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:10:06.163910 23125 catalog_manager.cc:1347] Generated new cluster ID: d3b3543a2c504b5dbf46dd768b003887
I20260620 08:10:06.163964 23125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:06.171486 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.159246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.12.132.254:38409)
0620 08:10:06.159837 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.159852 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.159923 (+ 71us) server_negotiation.cc:407] Connection header received
0620 08:10:06.160113 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.160122 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.160199 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.160422 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:06.161366 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.162822 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.163744 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.164355 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.164480 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.167129 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:06.167133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:06.167133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:06.167146 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:06.169841 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.170226 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.170229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.170229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.170351 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.170558 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.170559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.170559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.170890 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:06.171058 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.171127 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.171269 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":64,"threads_started":1}
I20260620 08:10:06.173700 23125 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:10:06.174602 23125 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:10:06.180558 23125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: Generated new TSK 0
I20260620 08:10:06.180747 23125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:10:06.217140 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:06.298547 23129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:06.298759 23129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:06.298784 23129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:06.299940 23129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:06.299983 23129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:06.300010 23129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:10:06.301545 23129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:06.301789 23129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:06.302026 23129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:06.307032 23134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.307024 23135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.307127 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.307341 23129 server_base.cc:1034] running on GCE node
I20260620 08:10:06.307536 23129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:06.307734 23129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:06.308885 23129 hybrid_clock.cc:648] HybridClock initialized: now 1781943006308865 us; error 42 us; skew 500 ppm
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:06.312206 23129 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:10:06.312742 23129 webserver.cc:460] Webserver started at http://127.12.132.193:36697/ using document root <none> and password file <none>
I20260620 08:10:06.312987 23129 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:06.313056 23129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:06.313210 23129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:06.314251 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "a210aac141424b089442bea514f71ada"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "c9a006decb0c211967f3639b5fa984b1"
server_key_iv: "e7f5e38a530e2014bd268bf080120c14"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.314723 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "a210aac141424b089442bea514f71ada"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "c9a006decb0c211967f3639b5fa984b1"
server_key_iv: "e7f5e38a530e2014bd268bf080120c14"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.316082 23129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:10:06.316998 23144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.317447 23129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:10:06.317670 23129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "a210aac141424b089442bea514f71ada"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "c9a006decb0c211967f3639b5fa984b1"
server_key_iv: "e7f5e38a530e2014bd268bf080120c14"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.317768 23129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:06.329965 23129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:06.332072 23129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:06.332240 23129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:06.332566 23129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:06.332904 23129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:06.332949 23129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.332973 23129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:06.332985 23129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.339298 23129 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35705
I20260620 08:10:06.339358 23257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35705 every 8 connection(s)
I20260620 08:10:06.339690 23129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:10:06.342190 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 23129
I20260620 08:10:06.342386 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:10:06.342639 12819 external_mini_cluster.cc:1428] Setting key e38a2cf4e1260b334dd949b17583ae9b
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:06.350651 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.341005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:56609 (local address 127.12.132.254:38409)
0620 08:10:06.341138 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.341140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.341313 (+ 173us) server_negotiation.cc:407] Connection header received
0620 08:10:06.341854 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.341855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.341869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.341929 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:06.343233 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.343780 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.344850 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.345055 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.345142 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.347664 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:06.347666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:06.347666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:06.347674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:06.349253 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.349689 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.349690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.349691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.349750 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.350003 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.350004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.350004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.350117 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:06.350193 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.350477 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.350559 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260620 08:10:06.350855 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.340961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38409 (local address 127.12.132.193:56609)
0620 08:10:06.341175 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.341219 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.341743 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.342140 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.342144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.342878 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:06.343136 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.343141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.343878 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.343879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.344748 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.344751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.345152 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.345154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.345272 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.345815 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:06.345823 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:06.347525 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:06.349361 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.349368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.349370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.349594 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.349819 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.349820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.349820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.349925 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.350185 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:06.350191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:06.350325 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.350453 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.350606 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
I20260620 08:10:06.351359 23258 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38409
I20260620 08:10:06.351524 23258 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:06.351737 23258 heartbeater.cc:511] Master 127.12.132.254:38409 requested a full tablet report, sending...
I20260620 08:10:06.352239 23067 ts_manager.cc:194] Registered new tserver with Master: a210aac141424b089442bea514f71ada (127.12.132.193:35705)
I20260620 08:10:06.352890 23067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:56609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:10:06.368044 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:06.448001 23263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:06.448201 23263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:06.448223 23263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:06.449393 23263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:06.449443 23263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:06.449469 23263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:10:06.450901 23263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:06.451131 23263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:06.451361 23263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:06.456689 23268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.456689 23269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.456904 23263 server_base.cc:1034] running on GCE node
W20260620 08:10:06.456713 23271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.457167 23263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:06.457408 23263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:06.458588 23263 hybrid_clock.cc:648] HybridClock initialized: now 1781943006458559 us; error 43 us; skew 500 ppm
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:06.461431 23263 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:10:06.461791 23263 webserver.cc:460] Webserver started at http://127.12.132.194:36463/ using document root <none> and password file <none>
I20260620 08:10:06.461978 23263 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:06.462019 23263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:06.462105 23263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:06.463075 23263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "282a5a1520864e1aba4afb2ca12f86b0"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "99fd74126d4d506d22264196161cfb7d"
server_key_iv: "9c2596ef3f615c8d4645327c8afbc9ff"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.463372 23263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "282a5a1520864e1aba4afb2ca12f86b0"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "99fd74126d4d506d22264196161cfb7d"
server_key_iv: "9c2596ef3f615c8d4645327c8afbc9ff"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.465004 23263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:10:06.465684 23278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.465883 23263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:10:06.465950 23263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "282a5a1520864e1aba4afb2ca12f86b0"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "99fd74126d4d506d22264196161cfb7d"
server_key_iv: "9c2596ef3f615c8d4645327c8afbc9ff"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.465996 23263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:06.484156 23263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:06.486511 23263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:06.486621 23263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:06.486788 23263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:06.487098 23263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:06.487131 23263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.487154 23263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:06.487203 23263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.493535 23263 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:35725
I20260620 08:10:06.493696 23391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:35725 every 8 connection(s)
I20260620 08:10:06.493958 23263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:06.503319 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 23263
I20260620 08:10:06.503427 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:10:06.503677 12819 external_mini_cluster.cc:1428] Setting key b3d75e3847677a47080c6bbc3c36d157
I20260620 08:10:06.503782 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.494899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:57865 (local address 127.12.132.254:38409)
0620 08:10:06.494983 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.494985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.495222 (+ 237us) server_negotiation.cc:407] Connection header received
0620 08:10:06.495759 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.495760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.495775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.495816 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:06.496832 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.497336 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.498298 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.498500 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.498585 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.500891 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:06.500892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:06.500893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:06.500899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:06.502360 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.502766 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.502767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.502767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.502815 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.503021 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.503022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.503022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.503118 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:06.503168 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.503599 (+ 431us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.503681 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 08:10:06.504280 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.494891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38409 (local address 127.12.132.194:57865)
0620 08:10:06.495089 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.495124 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.495644 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.495894 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.495898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.496450 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:06.496722 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.496727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.497428 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.497429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.498196 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.498197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.498593 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.498594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.498714 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.499213 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:06.499221 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:06.500776 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:06.502463 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.502469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.502470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.502696 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.502875 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.502875 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.502875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.502967 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.503203 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:06.503211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:06.503395 (+ 184us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.503903 (+ 508us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.504077 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
I20260620 08:10:06.504739 23392 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38409
I20260620 08:10:06.504856 23392 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:06.505029 23392 heartbeater.cc:511] Master 127.12.132.254:38409 requested a full tablet report, sending...
I20260620 08:10:06.505448 23067 ts_manager.cc:194] Registered new tserver with Master: 282a5a1520864e1aba4afb2ca12f86b0 (127.12.132.194:35725)
I20260620 08:10:06.505919 23067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:57865
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:10:06.528679 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:06.611366 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:06.611565 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:06.611588 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:06.612663 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:06.612699 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:06.612725 23397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:10:06.614291 23397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:06.614482 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:06.614645 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:06.620024 23402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.620056 23405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:06.620272 23403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:06.620545 23397 server_base.cc:1034] running on GCE node
I20260620 08:10:06.620772 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:06.621038 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:06.622215 23397 hybrid_clock.cc:648] HybridClock initialized: now 1781943006622198 us; error 46 us; skew 500 ppm
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:06.625171 23397 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:10:06.625842 23397 webserver.cc:460] Webserver started at http://127.12.132.195:46501/ using document root <none> and password file <none>
I20260620 08:10:06.626137 23397 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:06.626217 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:06.626386 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:10:06.627619 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "d5d8245576784521ae2406cfd997b3fd"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "170c721e93b9fbb81af3695fbfa6af91"
server_key_iv: "7ce5cfa0222cd15a8dd6cecb803e84eb"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.628154 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "d5d8245576784521ae2406cfd997b3fd"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "170c721e93b9fbb81af3695fbfa6af91"
server_key_iv: "7ce5cfa0222cd15a8dd6cecb803e84eb"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.629889 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:10:06.630681 23412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.630869 23397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:10:06.630986 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "d5d8245576784521ae2406cfd997b3fd"
format_stamp: "Formatted at 2026-06-20 08:10:06 on dist-test-slave-z0sp"
server_key: "170c721e93b9fbb81af3695fbfa6af91"
server_key_iv: "7ce5cfa0222cd15a8dd6cecb803e84eb"
server_key_version: "encryptionkey@0"
I20260620 08:10:06.631074 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:06.645558 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:06.647836 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:06.648000 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:06.648233 23397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:06.648617 23397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:06.648669 23397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.648710 23397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:06.648738 23397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:06.654579 23397 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:45029
I20260620 08:10:06.654618 23525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:45029 every 8 connection(s)
I20260620 08:10:06.654929 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943006, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:06.664139 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 23397
I20260620 08:10:06.664276 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:10:06.664480 12819 external_mini_cluster.cc:1428] Setting key 3d265834b993d19230d94375958c85bb
I20260620 08:10:06.666607 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.655754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:57081 (local address 127.12.132.254:38409)
0620 08:10:06.655799 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.655801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.655981 (+ 180us) server_negotiation.cc:407] Connection header received
0620 08:10:06.656626 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.656633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.656651 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.656741 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:06.658091 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.658720 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.660184 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.660454 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.660613 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.662985 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:06.662987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:06.662988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:06.662998 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:06.664951 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.665524 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.665525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.665525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.665594 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.666009 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.666009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.666010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.666123 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:06.666171 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.666475 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.666545 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:10:06.666839 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.655707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38409 (local address 127.12.132.195:57081)
0620 08:10:06.655856 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.655895 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.656374 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.656920 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.656926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.657681 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:06.657941 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.657947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.658857 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.658858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.660010 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.660015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.660637 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.660638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.660749 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.661481 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:06.661489 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:06.662883 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:06.665026 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.665033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.665035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.665449 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.665724 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:06.665726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:06.665727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:06.665923 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:06.666231 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:06.666237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:06.666362 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.666450 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.666570 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260620 08:10:06.667254 23526 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38409
I20260620 08:10:06.667354 23526 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:06.667526 23526 heartbeater.cc:511] Master 127.12.132.254:38409 requested a full tablet report, sending...
I20260620 08:10:06.667909 23067 ts_manager.cc:194] Registered new tserver with Master: d5d8245576784521ae2406cfd997b3fd (127.12.132.195:45029)
I20260620 08:10:06.668292 23067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:57081
I20260620 08:10:06.676420 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:10:06.683800 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.677930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50608 (local address 127.12.132.254:38409)
0620 08:10:06.677974 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.677976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.678056 (+ 80us) server_negotiation.cc:407] Connection header received
0620 08:10:06.678100 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.678101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.678110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.678157 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:06.678651 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.679125 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.679836 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.680030 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.680092 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.681361 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:06.681362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:06.681362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:06.681368 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:06.682844 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.683159 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.683160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.683160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.683208 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:06.683367 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:06.683368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:06.683368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:06.683497 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:06.683568 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.683680 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.683738 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:10:06.685611 23067 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:10:06.688045 23067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:10:06.694430 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.12.132.194:35725)
0620 08:10:06.691095 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.691098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.691107 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:10:06.691144 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.691146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.691159 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.691234 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:06.691333 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.692355 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.693498 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694021 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694120 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694177 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.694301 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.694342 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":89,"threads_started":1}
I20260620 08:10:06.694643 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.12.132.193:35705)
0620 08:10:06.690812 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.690817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.690827 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:10:06.690863 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.690865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.690881 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.690979 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:06.691185 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.691921 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.693337 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694065 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694139 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694187 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.694518 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.694566 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":74,"threads_started":1}
I20260620 08:10:06.694731 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.12.132.195:45029)
0620 08:10:06.690787 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.690792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.690801 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:10:06.690840 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.690842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.690859 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.690972 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:06.691145 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.692269 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.693606 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694280 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694346 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694388 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.694609 (+ 221us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.694648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260620 08:10:06.694793 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:45029 (local address 127.0.0.1:33620)
0620 08:10:06.690447 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.690458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.690523 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.690978 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.690981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.690984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:06.691058 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.691060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.692381 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.692382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.693523 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.693525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694419 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.694490 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694493 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.694545 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.694575 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":55,"thread_start_us":86,"threads_started":1}
I20260620 08:10:06.694871 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35705 (local address 127.0.0.1:48726)
0620 08:10:06.690414 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.690456 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.690523 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.691052 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.691053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.691055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:06.691108 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.691110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.692019 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.692021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.693252 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.693255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694223 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.694287 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694291 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.694338 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.694379 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260620 08:10:06.694463 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.690497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35725 (local address 127.0.0.1:47618)
0620 08:10:06.690691 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.690701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.690752 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.691236 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.691237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.691238 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:06.691269 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.691270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.692432 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.692433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.693407 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.693409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.694120 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.694121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.694195 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.694208 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.694279 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.694331 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":59,"threads_started":1}
I20260620 08:10:06.695631 23460 tablet_service.cc:1467] Processing CreateTablet for tablet f91ea695194f479a9d324c2a0ed4691b (DEFAULT_TABLE table=test-table [id=944924c515d1447193e1a613337e1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:06.695737 23326 tablet_service.cc:1467] Processing CreateTablet for tablet f91ea695194f479a9d324c2a0ed4691b (DEFAULT_TABLE table=test-table [id=944924c515d1447193e1a613337e1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:06.695993 23460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91ea695194f479a9d324c2a0ed4691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:06.696089 23326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91ea695194f479a9d324c2a0ed4691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:06.696913 23192 tablet_service.cc:1467] Processing CreateTablet for tablet f91ea695194f479a9d324c2a0ed4691b (DEFAULT_TABLE table=test-table [id=944924c515d1447193e1a613337e1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:06.697218 23192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91ea695194f479a9d324c2a0ed4691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:06.699297 23543 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Bootstrap starting.
I20260620 08:10:06.699699 23544 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Bootstrap starting.
I20260620 08:10:06.700206 23543 tablet_bootstrap.cc:654] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:06.700455 23544 tablet_bootstrap.cc:654] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:06.700619 23545 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: Bootstrap starting.
I20260620 08:10:06.701014 23543 log.cc:824] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:06.701081 23544 log.cc:824] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:06.701349 23545 tablet_bootstrap.cc:654] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:06.701851 23544 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: No bootstrap required, opened a new log
I20260620 08:10:06.701859 23545 log.cc:824] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:06.701881 23543 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: No bootstrap required, opened a new log
I20260620 08:10:06.701920 23544 ts_tablet_manager.cc:1397] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:10:06.701936 23543 ts_tablet_manager.cc:1397] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 08:10:06.702584 23545 tablet_bootstrap.cc:492] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: No bootstrap required, opened a new log
I20260620 08:10:06.702653 23545 ts_tablet_manager.cc:1397] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:10:06.703508 23544 raft_consensus.cc:348] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.703673 23544 raft_consensus.cc:374] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:06.703732 23544 raft_consensus.cc:729] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282a5a1520864e1aba4afb2ca12f86b0, State: Initialized, Role: FOLLOWER
I20260620 08:10:06.703843 23544 consensus_queue.cc:260] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.703879 23543 raft_consensus.cc:348] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.704036 23543 raft_consensus.cc:374] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:06.704080 23543 raft_consensus.cc:729] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d8245576784521ae2406cfd997b3fd, State: Initialized, Role: FOLLOWER
I20260620 08:10:06.704164 23544 ts_tablet_manager.cc:1428] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:10:06.704185 23543 consensus_queue.cc:260] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.704288 23392 heartbeater.cc:503] Master 127.12.132.254:38409 was elected leader, sending a full tablet report...
I20260620 08:10:06.704427 23543 ts_tablet_manager.cc:1428] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:10:06.704676 23526 heartbeater.cc:503] Master 127.12.132.254:38409 was elected leader, sending a full tablet report...
I20260620 08:10:06.704576 23545 raft_consensus.cc:348] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.704731 23545 raft_consensus.cc:374] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:06.704768 23545 raft_consensus.cc:729] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a210aac141424b089442bea514f71ada, State: Initialized, Role: FOLLOWER
I20260620 08:10:06.704895 23545 consensus_queue.cc:260] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.705174 23545 ts_tablet_manager.cc:1428] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260620 08:10:06.705242 23258 heartbeater.cc:503] Master 127.12.132.254:38409 was elected leader, sending a full tablet report...
I20260620 08:10:06.713148 23551 raft_consensus.cc:482] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:10:06.713253 23551 raft_consensus.cc:504] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.713510 23551 leader_election.cc:288] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 282a5a1520864e1aba4afb2ca12f86b0 (127.12.132.194:35725), d5d8245576784521ae2406cfd997b3fd (127.12.132.195:45029)
I20260620 08:10:06.716872 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.713906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52729 (local address 127.12.132.194:35725)
0620 08:10:06.713951 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.713954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.714011 (+ 57us) server_negotiation.cc:407] Connection header received
0620 08:10:06.714081 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.714081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.714089 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.714158 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:06.714323 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.714798 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.715900 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.716370 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.716480 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.716493 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.716733 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.716786 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 08:10:06.716872 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.713858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35725 (local address 127.12.132.193:52729)
0620 08:10:06.713925 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.713938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.714018 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.714170 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.714172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.714174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:06.714249 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.714251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.714886 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.714887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.715674 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.715676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.716477 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.716478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.716632 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.716636 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.716723 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.716786 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:10:06.717118 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.713858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:45029 (local address 127.12.132.193:44299)
0620 08:10:06.714269 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.714279 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:06.714378 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:06.714524 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:06.714525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:06.714527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:06.714629 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.714631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.715298 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.715299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.716071 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:06.716073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.716803 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.716804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:06.716882 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.716886 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:10:06.716973 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 08:10:06.717019 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":207,"thread_start_us":150,"threads_started":1}
I20260620 08:10:06.717108 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.713889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:44299 (local address 127.12.132.195:45029)
0620 08:10:06.713940 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.713942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.714370 (+ 428us) server_negotiation.cc:407] Connection header received
0620 08:10:06.714439 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.714441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.714457 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.714521 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:06.714698 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.715210 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.716198 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.716706 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.716804 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.716813 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.716981 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.717042 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 08:10:06.717399 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91ea695194f479a9d324c2a0ed4691b" candidate_uuid: "a210aac141424b089442bea514f71ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282a5a1520864e1aba4afb2ca12f86b0" is_pre_election: true
I20260620 08:10:06.717595 23346 raft_consensus.cc:2454] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a210aac141424b089442bea514f71ada in term 0.
I20260620 08:10:06.717706 23480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91ea695194f479a9d324c2a0ed4691b" candidate_uuid: "a210aac141424b089442bea514f71ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d8245576784521ae2406cfd997b3fd" is_pre_election: true
I20260620 08:10:06.717895 23480 raft_consensus.cc:2454] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a210aac141424b089442bea514f71ada in term 0.
I20260620 08:10:06.717912 23145 leader_election.cc:302] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282a5a1520864e1aba4afb2ca12f86b0, a210aac141424b089442bea514f71ada; no voters:
I20260620 08:10:06.718173 23551 raft_consensus.cc:2781] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:10:06.718240 23551 raft_consensus.cc:482] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:10:06.718271 23551 raft_consensus.cc:3037] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:06.719135 23551 raft_consensus.cc:504] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.719318 23551 leader_election.cc:288] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [CANDIDATE]: Term 1 election: Requested vote from peers 282a5a1520864e1aba4afb2ca12f86b0 (127.12.132.194:35725), d5d8245576784521ae2406cfd997b3fd (127.12.132.195:45029)
I20260620 08:10:06.719492 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91ea695194f479a9d324c2a0ed4691b" candidate_uuid: "a210aac141424b089442bea514f71ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282a5a1520864e1aba4afb2ca12f86b0"
I20260620 08:10:06.719583 23346 raft_consensus.cc:3037] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:06.719533 23480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91ea695194f479a9d324c2a0ed4691b" candidate_uuid: "a210aac141424b089442bea514f71ada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d8245576784521ae2406cfd997b3fd"
I20260620 08:10:06.719785 23480 raft_consensus.cc:3037] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:06.720606 23346 raft_consensus.cc:2454] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a210aac141424b089442bea514f71ada in term 1.
I20260620 08:10:06.720692 23480 raft_consensus.cc:2454] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a210aac141424b089442bea514f71ada in term 1.
I20260620 08:10:06.720796 23145 leader_election.cc:302] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282a5a1520864e1aba4afb2ca12f86b0, a210aac141424b089442bea514f71ada; no voters:
I20260620 08:10:06.720937 23551 raft_consensus.cc:2781] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:06.721124 23551 raft_consensus.cc:686] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [term 1 LEADER]: Becoming Leader. State: Replica: a210aac141424b089442bea514f71ada, State: Running, Role: LEADER
I20260620 08:10:06.721235 23551 consensus_queue.cc:237] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } }
I20260620 08:10:06.722188 23067 catalog_manager.cc:5495] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada reported cstate change: term changed from 0 to 1, leader changed from <none> to a210aac141424b089442bea514f71ada (127.12.132.193). New cstate: current_term: 1 leader_uuid: "a210aac141424b089442bea514f71ada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a210aac141424b089442bea514f71ada" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 } health_report { overall_health: UNKNOWN } } }
I20260620 08:10:06.731101 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.728530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.12.132.193:35705)
0620 08:10:06.728603 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:06.728606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:06.728618 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:10:06.728650 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:06.728650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:06.728666 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:06.728720 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:10:06.728880 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.729331 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.730144 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:06.730338 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:06.730405 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:06.730573 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:10:06.730662 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:10:06.730722 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:06.730922 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 08:10:06.730970 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:10:06.733071 23480 raft_consensus.cc:1261] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 1 FOLLOWER]: Refusing update from remote peer a210aac141424b089442bea514f71ada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:10:06.733079 23346 raft_consensus.cc:1261] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 1 FOLLOWER]: Refusing update from remote peer a210aac141424b089442bea514f71ada: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:10:06.733445 23551 consensus_queue.cc:1035] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d8245576784521ae2406cfd997b3fd" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:10:06.733608 23553 consensus_queue.cc:1035] T f91ea695194f479a9d324c2a0ed4691b P a210aac141424b089442bea514f71ada [LEADER]: Connected to new peer: Peer: permanent_uuid: "282a5a1520864e1aba4afb2ca12f86b0" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:10:06.736255 23557 mvcc.cc:205] Tried to move back new op lower bound from 7298838555575762944 to 7298838555531198464. Current Snapshot: MvccSnapshot[applied={T|T < 7298838555575762944}]
I20260620 08:10:06.738466 23066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:10:06.738564 23066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:10:06.739377 23066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba13f9d1a67d4db4afa0c2670e3e4212: Sending DeleteTablet for 3 replicas of tablet f91ea695194f479a9d324c2a0ed4691b
I20260620 08:10:06.739687 23192 tablet_service.cc:1514] Processing DeleteTablet for tablet f91ea695194f479a9d324c2a0ed4691b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:10:06 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:48726
I20260620 08:10:06.739718 23326 tablet_service.cc:1514] Processing DeleteTablet for tablet f91ea695194f479a9d324c2a0ed4691b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:10:06 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:47618
I20260620 08:10:06.739714 23460 tablet_service.cc:1514] Processing DeleteTablet for tablet f91ea695194f479a9d324c2a0ed4691b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:10:06 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:33620
I20260620 08:10:06.739926 23567 tablet_replica.cc:330] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: stopping tablet replica
I20260620 08:10:06.740028 23567 raft_consensus.cc:2229] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:10:06.740124 23567 raft_consensus.cc:2258] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:10:06.740257 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 23129
I20260620 08:10:06.740998 23567 ts_tablet_manager.cc:1905] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:10:06.741480 23568 tablet_replica.cc:330] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: stopping tablet replica
I20260620 08:10:06.741593 23568 raft_consensus.cc:2229] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:10:06.741693 23568 raft_consensus.cc:2258] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:10:06.742141 23568 ts_tablet_manager.cc:1905] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:10:06.742645 23567 ts_tablet_manager.cc:1918] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:10:06.742718 23567 log.cc:1196] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/f91ea695194f479a9d324c2a0ed4691b
I20260620 08:10:06.743008 23567 ts_tablet_manager.cc:1939] T f91ea695194f479a9d324c2a0ed4691b P 282a5a1520864e1aba4afb2ca12f86b0: Deleting consensus metadata
I20260620 08:10:06.743556 23051 catalog_manager.cc:4841] TS 282a5a1520864e1aba4afb2ca12f86b0 (127.12.132.194:35725): tablet f91ea695194f479a9d324c2a0ed4691b (table test-table [id=944924c515d1447193e1a613337e1768]) successfully deleted
I20260620 08:10:06.743763 23568 ts_tablet_manager.cc:1918] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:10:06.743816 23568 log.cc:1196] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/f91ea695194f479a9d324c2a0ed4691b
I20260620 08:10:06.744071 23568 ts_tablet_manager.cc:1939] T f91ea695194f479a9d324c2a0ed4691b P d5d8245576784521ae2406cfd997b3fd: Deleting consensus metadata
I20260620 08:10:06.744606 23052 catalog_manager.cc:4841] TS d5d8245576784521ae2406cfd997b3fd (127.12.132.195:45029): tablet f91ea695194f479a9d324c2a0ed4691b (table test-table [id=944924c515d1447193e1a613337e1768]) successfully deleted
W20260620 08:10:06.747380 23051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:35705 (error 108)
I20260620 08:10:06.747408 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 23263
I20260620 08:10:06.752130 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 23397
I20260620 08:10:06.752370 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:06.752180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35705 (local address 127.0.0.1:48736)
0620 08:10:06.752273 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:06.752329 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260620 08:10:06.754155 23051 catalog_manager.cc:4584] TS a210aac141424b089442bea514f71ada (127.12.132.193:35705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f91ea695194f479a9d324c2a0ed4691b: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35705: connect: Connection refused (error 111)
I20260620 08:10:06.758141 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 23035
2026-06-20T08:10:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1930 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23572](info): set up 2 sockets
Jun 20 08:10:06 dist-test-slave-z0sp krb5kdc[23572](info): commencing operation
krb5kdc: starting...
W20260620 08:10:07.806137 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:10:07 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:10:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:10:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangeradmin_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangeradmin_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangerlookup_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangerlookup_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
I20260620 08:10:07.987794 12819 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZ6VfL9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/postgres -l logfile start
2026-06-20 08:10:10.729 UTC [23602] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 08:10:10.729 UTC [23602] LOG: listening on IPv4 address "127.12.132.212", port 55359
2026-06-20 08:10:10.733 UTC [23602] LOG: listening on Unix socket "/tmp/.s.PGSQL.55359"
2026-06-20 08:10:10.771 UTC [23605] LOG: database system was shut down at 2026-06-20 08:10:09 UTC
2026-06-20 08:10:10.775 UTC [23602] LOG: database system is ready to accept connections
I20260620 08:10:11.742287 12819 mini_postgres.cc:96] Postgres bound to 55359
2026-06-20 08:10:11.748 UTC [23613] FATAL: database "slave" does not exist
127.12.132.212:55359 - accepting connections
I20260620 08:10:11.749065 12819 mini_ranger.cc:161] Starting Ranger...
I20260620 08:10:11.764572 12819 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 08:10:12.315179 12819 mini_ranger.cc:87] Created ranger Postgres database
I20260620 08:10:12.315254 12819 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin
2026-06-20 08:10:12,678 [I] DB FLAVOR :POSTGRES
2026-06-20 08:10:12,679 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 08:10:12,679 [I] Checking connection..
2026-06-20 08:10:12,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 08:10:12,946 [I] Checking connection passed.
2026-06-20 08:10:12,946 [I] --------- Verifying version history table ---------
2026-06-20 08:10:12,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:10:13,192 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 08:10:13,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:10:13,424 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 08:10:13,425 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 08:10:13,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 08:10:13.655 UTC [23677] WARNING: there is no transaction in progress
2026-06-20 08:10:13,662 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 08:10:13,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:10:13,913 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 08:10:13,913 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 08:10:13,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 08:10:14,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 08:10:14,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z0sp.c.gcp-upstream.internal','N') ;"
2026-06-20 08:10:14,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 08:10:14,895 [I] Table x_portal_user does not exist in database ranger
2026-06-20 08:10:14,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 08:10:15,155 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 08:10:15,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 08:10:15,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 08:10:15,672 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 08:10:15,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 08:10:16.698 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.717 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.734 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.750 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.767 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.784 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.842 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.849 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.866 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.876 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.886 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.901 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.912 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.918 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.927 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:16.935 UTC [23785] WARNING: there is no transaction in progress
2026-06-20 08:10:17,643 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 08:10:17,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 08:10:17,876 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 08:10:17,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 08:10:18,114 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 08:10:18,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 08:10:18,387 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 08:10:18,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 08:10:18,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 08:10:18,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z0sp.c.gcp-upstream.internal';"
2026-06-20 08:10:19,162 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 08:10:19,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 08:10:19,430 [I] DB_PATCHES have already been applied
I20260620 08:10:19.436399 12819 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZ6VfL9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 08:10:19.436494 12819 mini_ranger.cc:193] Using host: 127.12.132.212
I20260620 08:10:19.439772 12819 mini_ranger.cc:227] Ranger admin URL: http://127.12.132.212:33511
Jun 20, 2026 8:10:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:10:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:10:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 8:10:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 8:10:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 8:10:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 8:10:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 08:10:20.556901 12819 mini_ranger.cc:160] Time spent starting Ranger: real 8.808s user 0.000s sys 0.010s
Jun 20, 2026 8:10:42 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 8:10:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 8:10:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 8:10:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 8:10:43 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 08:10:44.128132 12819 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangerkms_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/rangerkms_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.132.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.132.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 08:10:44 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943044, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 08:10:44.230530 12819 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 08:10:44.243106 12819 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 08:10:45.264853 12819 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 08:10:45.264971 12819 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms
I20260620 08:10:45.264991 12819 mini_ranger_kms.cc:227] Using postgres at 127.12.132.212:55359
2026-06-20 08:10:45,693 [I] DB FLAVOR :POSTGRES
2026-06-20 08:10:45,694 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 08:10:45,694 [I] Checking connection
2026-06-20 08:10:45,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 08:10:46,050 [I] connection success
2026-06-20 08:10:46,050 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 08:10:46,050 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 08:10:46,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 08:10:46,348 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 08:10:46,348 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 08:10:46,349 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 08:10:46,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:55359/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 08:10:46,675 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 08:10:47.180924 12819 mini_ranger_kms.cc:310] Created kms service
I20260620 08:10:47.356978 12819 mini_ranger_kms.cc:326] Created kudu user
I20260620 08:10:47.403252 12819 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 08:10:47 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943044, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:10:47.881009 12819 mini_ranger_kms.cc:384] Added ranger policy
I20260620 08:10:47.881139 12819 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 08:10:47.881165 12819 mini_ranger_kms.cc:240] Using host: 127.12.132.212
I20260620 08:10:47.884913 12819 mini_ranger_kms.cc:276] Ranger KMS PID: 23966
I20260620 08:10:47.885031 12819 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.132.212:50159
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 8:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 08:10:50.235952 12819 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.005s user 0.017s sys 0.370s
I20260620 08:10:50.236063 12819 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 08:10:50.236153 12819 mini_ranger_kms.cc:401] 127.12.132.212:50159/kms/v1/keys
Jun 20 08:10:50 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781943050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 08:10:51 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
Jun 20 08:10:51 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943050, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
Jun 20 08:10:54 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:10:54.096050 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43243
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43243
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:50159/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:54.241068 24016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:54.241276 24016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:54.241307 24016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:54.242542 24016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:10:54.242589 24016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:54.242604 24016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:54.242617 24016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:10:54.242630 24016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:10:54.244367 24016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.132.212:50159/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43243
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:54.244619 24016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:54.244827 24016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:10:54.250186 24016 server_base.cc:1034] running on GCE node
W20260620 08:10:54.250491 24021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:54.250501 24022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:54.250494 24024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:54.251097 24016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:54.251423 24016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:54.252599 24016 hybrid_clock.cc:648] HybridClock initialized: now 1781943054252589 us; error 45 us; skew 500 ppm
Jun 20 08:10:54 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943054, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:54.259874 24016 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:10:54.260300 24016 webserver.cc:460] Webserver started at http://127.12.132.254:42727/ using document root <none> and password file <none>
I20260620 08:10:54.260565 24016 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:54.260627 24016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:54.260761 24016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:10:54 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943054, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:10:54.309922 24016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "b693e2ac71fe4211872c8581ca7998a5"
format_stamp: "Formatted at 2026-06-20 08:10:54 on dist-test-slave-z0sp"
server_key: "1dd5546bfed4244fd009ee09d07aceab"
server_key_iv: "f9f718f20b72dbb611a8fdbddab8fd16"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:54.310412 24016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "b693e2ac71fe4211872c8581ca7998a5"
format_stamp: "Formatted at 2026-06-20 08:10:54 on dist-test-slave-z0sp"
server_key: "1dd5546bfed4244fd009ee09d07aceab"
server_key_iv: "f9f718f20b72dbb611a8fdbddab8fd16"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:54.312054 24016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:10:54.324981 24031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:54.325192 24016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:10:54.325299 24016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "b693e2ac71fe4211872c8581ca7998a5"
format_stamp: "Formatted at 2026-06-20 08:10:54 on dist-test-slave-z0sp"
server_key: "1dd5546bfed4244fd009ee09d07aceab"
server_key_iv: "f9f718f20b72dbb611a8fdbddab8fd16"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:54.325387 24016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:54.343539 24016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:54.345727 24016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:54.345906 24016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:54.352528 24016 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:43243
I20260620 08:10:54.352550 24083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:43243 every 8 connection(s)
I20260620 08:10:54.352908 24016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:10:54.353662 24084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:54.355726 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 24016
I20260620 08:10:54.355865 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:10:54.356073 12819 external_mini_cluster.cc:1428] Setting key 37ff7e41d4fe0e65fa23c423fa50e481
I20260620 08:10:54.356606 24084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: Bootstrap starting.
I20260620 08:10:54.357324 24084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:54.358098 24084 log.cc:824] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:54.359434 24084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: No bootstrap required, opened a new log
I20260620 08:10:54.362825 24084 raft_consensus.cc:348] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } }
I20260620 08:10:54.363082 24084 raft_consensus.cc:374] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:54.363123 24084 raft_consensus.cc:729] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693e2ac71fe4211872c8581ca7998a5, State: Initialized, Role: FOLLOWER
I20260620 08:10:54.363344 24084 consensus_queue.cc:260] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } }
I20260620 08:10:54.363448 24084 raft_consensus.cc:388] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:10:54.363520 24084 raft_consensus.cc:482] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:10:54.363603 24084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 0 FOLLOWER]: Advancing to term 1
Jun 20 08:10:54 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:54.364532 24084 raft_consensus.cc:504] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } }
I20260620 08:10:54.364668 24084 leader_election.cc:302] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b693e2ac71fe4211872c8581ca7998a5; no voters:
I20260620 08:10:54.364908 24084 leader_election.cc:288] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:10:54.365252 24084 sys_catalog.cc:564] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:10:54.364966 24089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:54.365854 24089 raft_consensus.cc:686] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [term 1 LEADER]: Becoming Leader. State: Replica: b693e2ac71fe4211872c8581ca7998a5, State: Running, Role: LEADER
I20260620 08:10:54.366042 24089 consensus_queue.cc:237] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } }
I20260620 08:10:54.366452 24089 sys_catalog.cc:455] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b693e2ac71fe4211872c8581ca7998a5. Latest consensus state: current_term: 1 leader_uuid: "b693e2ac71fe4211872c8581ca7998a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } } }
I20260620 08:10:54.366626 24089 sys_catalog.cc:458] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:54.366827 24084 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 08:10:54.367141 24090 sys_catalog.cc:455] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b693e2ac71fe4211872c8581ca7998a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b693e2ac71fe4211872c8581ca7998a5" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43243 } } }
I20260620 08:10:54.367298 24090 sys_catalog.cc:458] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5 [sys.catalog]: This master's current role is: LEADER
I20260620 08:10:54.370679 24103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:10:54.372639 24103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:10:54.374073 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:54.356636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.12.132.254:43243)
0620 08:10:54.356833 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:54.356843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:54.356873 (+ 30us) server_negotiation.cc:407] Connection header received
0620 08:10:54.357810 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:54.357817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:54.357865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:54.358064 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:54.358948 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:54.360434 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:54.361280 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:54.361729 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:54.361862 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:54.364459 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:54.364463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:54.364464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:54.364483 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:54.366729 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:54.367142 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:54.367144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:54.367145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:54.367265 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:54.367471 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:54.367472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:54.367472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:54.370091 (+ 2619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:54.371139 (+ 1048us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:54.371303 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 08:10:54.373882 (+ 2579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
I20260620 08:10:54.386245 24103 catalog_manager.cc:1347] Generated new cluster ID: deec8444ff8149abb891f524a4051469
I20260620 08:10:54.386332 24103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:10:54.401320 24103 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:10:54.402249 24103 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:10:54.409718 24103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: Generated new TSK 0
I20260620 08:10:54.409999 24103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:10:55.179764 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:50159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:55.288244 24129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:55.288591 24129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:55.288623 24129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:55.289865 24129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:55.289906 24129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:55.289933 24129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:10:55.291489 24129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.132.212:50159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:55.291738 24129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:55.292011 24129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:55.298209 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.298302 24137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.298324 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:55.298600 24129 server_base.cc:1034] running on GCE node
I20260620 08:10:55.298813 24129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:55.299088 24129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:55.300423 24129 hybrid_clock.cc:648] HybridClock initialized: now 1781943055300409 us; error 29 us; skew 500 ppm
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:55.303160 24129 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:10:55.303658 24129 webserver.cc:460] Webserver started at http://127.12.132.193:35945/ using document root <none> and password file <none>
I20260620 08:10:55.304042 24129 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:55.304114 24129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:55.304255 24129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:10:55.317453 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "35e237f6172647779e0dfe91d80616dc"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "282f2bcd437c9d0252e719279057b6a6"
server_key_iv: "80f86a69d2b7123b0d48b77e292d06f0"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.317865 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "35e237f6172647779e0dfe91d80616dc"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "282f2bcd437c9d0252e719279057b6a6"
server_key_iv: "80f86a69d2b7123b0d48b77e292d06f0"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.319725 24129 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260620 08:10:55.346230 24144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.346884 24129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:10:55.347090 24129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "35e237f6172647779e0dfe91d80616dc"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "282f2bcd437c9d0252e719279057b6a6"
server_key_iv: "80f86a69d2b7123b0d48b77e292d06f0"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.347463 24129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:55.374133 24129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:55.376614 24129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:55.376881 24129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:55.377210 24129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:55.377633 24129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:55.377732 24129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.377808 24129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:55.377863 24129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.393316 24129 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:45289
I20260620 08:10:55.393962 24129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:10:55.394905 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:45289 every 8 connection(s)
I20260620 08:10:55.402865 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 24129
I20260620 08:10:55.402998 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:10:55.403199 12819 external_mini_cluster.cc:1428] Setting key 020501e76956b72878cd330dba7d9c8c
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:55.409328 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.394956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43243 (local address 127.12.132.193:47849)
0620 08:10:55.395270 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:55.395326 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:55.396429 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:55.396938 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:55.396942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:55.397409 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:55.397828 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:55.397835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.398984 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.398985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:55.400612 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:55.400616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.401271 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.401272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:55.401512 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:55.402233 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:55.402243 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:55.404622 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:55.406595 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:55.406611 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:55.406614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:55.407058 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:55.407314 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:55.407316 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:55.407316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:55.407444 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:55.408552 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:55.408559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:55.408736 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 08:10:55.408850 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 08:10:55.409004 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":181,"threads_started":1}
I20260620 08:10:55.409835 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.396144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:47849 (local address 127.12.132.254:43243)
0620 08:10:55.396413 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:55.396415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:55.396429 (+ 14us) server_negotiation.cc:407] Connection header received
0620 08:10:55.396664 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:55.396665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:55.396679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:55.396797 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:55.397948 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.398875 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.400719 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.401118 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.401192 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:55.404757 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:55.404758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:55.404758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:55.404765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:55.406456 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:55.407146 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:55.407148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:55.407148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:55.407218 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:55.407524 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:55.407525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:55.407526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:55.407694 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:55.407787 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:55.409619 (+ 1832us) server_negotiation.cc:299] Negotiation successful
0620 08:10:55.409708 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":115,"threads_started":1}
I20260620 08:10:55.412647 24258 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43243
I20260620 08:10:55.412774 24258 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:55.413007 24258 heartbeater.cc:511] Master 127.12.132.254:43243 requested a full tablet report, sending...
I20260620 08:10:55.413640 24048 ts_manager.cc:194] Registered new tserver with Master: 35e237f6172647779e0dfe91d80616dc (127.12.132.193:45289)
I20260620 08:10:55.414752 24048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:47849
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:10:55.441146 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:50159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:55.565743 24266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:55.566025 24266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:55.566064 24266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:55.567821 24266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:55.567881 24266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:55.567922 24266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:10:55.570174 24266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.132.212:50159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:55.570425 24266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:55.570796 24266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:55.577891 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.577919 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.578434 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:55.579311 24266 server_base.cc:1034] running on GCE node
I20260620 08:10:55.579571 24266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:55.579865 24266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:55.581046 24266 hybrid_clock.cc:648] HybridClock initialized: now 1781943055581026 us; error 34 us; skew 500 ppm
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:55.584916 24266 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:10:55.585649 24266 webserver.cc:460] Webserver started at http://127.12.132.194:44247/ using document root <none> and password file <none>
I20260620 08:10:55.586066 24266 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:55.586133 24266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:55.586411 24266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:10:55.604060 24266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "f81f11222a9c4b42bde55070bff13a97"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "e2978baba826b9252704328fe9fb427c"
server_key_iv: "33e4acad81592f282dd4c97637040b3a"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.605034 24266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "f81f11222a9c4b42bde55070bff13a97"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "e2978baba826b9252704328fe9fb427c"
server_key_iv: "33e4acad81592f282dd4c97637040b3a"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.611186 24266 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260620 08:10:55.631078 24283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.631422 24266 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260620 08:10:55.631536 24266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "f81f11222a9c4b42bde55070bff13a97"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "e2978baba826b9252704328fe9fb427c"
server_key_iv: "33e4acad81592f282dd4c97637040b3a"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.631675 24266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:55.645541 24266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:55.648759 24266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:55.648981 24266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:55.649328 24266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:55.649812 24266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:55.649896 24266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.649967 24266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:55.650024 24266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.660905 24266 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:38365
I20260620 08:10:55.661532 24266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:10:55.662590 24396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:38365 every 8 connection(s)
I20260620 08:10:55.669546 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 24266
I20260620 08:10:55.669679 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:10:55.669919 12819 external_mini_cluster.cc:1428] Setting key c8bda181820c930f0d2e18a5c3d16856
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:55.678730 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.663673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:47037 (local address 127.12.132.254:43243)
0620 08:10:55.663717 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:55.663719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:55.664136 (+ 417us) server_negotiation.cc:407] Connection header received
0620 08:10:55.665100 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:55.665101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:55.665116 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:55.665145 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:55.666371 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.667330 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.669379 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.669751 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.669805 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:55.673615 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:55.673616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:55.673617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:55.673626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:55.675435 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:55.676030 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:55.676032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:55.676032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:55.676101 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:55.676659 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:55.676661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:55.676662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:55.677988 (+ 1326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:55.678067 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:55.678497 (+ 430us) server_negotiation.cc:299] Negotiation successful
0620 08:10:55.678630 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:10:55.679569 24399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.663269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43243 (local address 127.12.132.194:47037)
0620 08:10:55.663986 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:55.664029 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:55.664982 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:55.665317 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:55.665320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:55.665709 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:55.666255 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:55.666262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.667582 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.667584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:55.669240 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:55.669245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.670068 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.670069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:55.670235 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:55.671194 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:55.671208 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:55.673429 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:55.675547 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:55.675555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:55.675558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:55.675928 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:55.676174 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:55.676175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:55.676175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:55.676525 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:55.678113 (+ 1588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:55.678121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:55.678304 (+ 183us) client_negotiation.cc:769] Sending connection context
0620 08:10:55.679104 (+ 800us) client_negotiation.cc:240] Negotiation successful
0620 08:10:55.679309 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":201,"threads_started":1}
I20260620 08:10:55.680120 24397 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43243
I20260620 08:10:55.680219 24397 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:55.680433 24397 heartbeater.cc:511] Master 127.12.132.254:43243 requested a full tablet report, sending...
I20260620 08:10:55.680869 24048 ts_manager.cc:194] Registered new tserver with Master: f81f11222a9c4b42bde55070bff13a97 (127.12.132.194:38365)
I20260620 08:10:55.681366 24048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:47037
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:10:55.715433 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:50159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:10:55.876689 24402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:10:55.877105 24402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:10:55.877218 24402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:10:55.880358 24402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:10:55.880528 24402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:10:55.880604 24402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:10:55.885133 24402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.132.212:50159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:10:55.885712 24402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:10:55.886174 24402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:10:55.897441 24409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.897516 24411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:10:55.897814 24408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:10:55.898077 24402 server_base.cc:1034] running on GCE node
I20260620 08:10:55.898375 24402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:10:55.898715 24402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:10:55.899955 24402 hybrid_clock.cc:648] HybridClock initialized: now 1781943055899914 us; error 36 us; skew 500 ppm
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:10:55.904850 24402 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:10:55.905586 24402 webserver.cc:460] Webserver started at http://127.12.132.195:40199/ using document root <none> and password file <none>
I20260620 08:10:55.905993 24402 fs_manager.cc:359] Metadata directory not provided
I20260620 08:10:55.906154 24402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:10:55.906471 24402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:10:55.930433 24402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "fb8e093df5134ff4900edfdaf092ccf3"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "f221b615679cef561774fde693f242ce"
server_key_iv: "f49e0ddc6b0b84cf3eb504fa16212444"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.930948 24402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "fb8e093df5134ff4900edfdaf092ccf3"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "f221b615679cef561774fde693f242ce"
server_key_iv: "f49e0ddc6b0b84cf3eb504fa16212444"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.933326 24402 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260620 08:10:55.957634 24418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.957885 24402 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.002s
I20260620 08:10:55.957976 24402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "fb8e093df5134ff4900edfdaf092ccf3"
format_stamp: "Formatted at 2026-06-20 08:10:55 on dist-test-slave-z0sp"
server_key: "f221b615679cef561774fde693f242ce"
server_key_iv: "f49e0ddc6b0b84cf3eb504fa16212444"
server_key_version: "kuduclusterkey@0"
I20260620 08:10:55.958068 24402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:10:55.981029 24402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:10:55.983453 24402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:10:55.983629 24402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:10:55.983855 24402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:10:55.984234 24402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:10:55.984280 24402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.984310 24402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:10:55.984338 24402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:10:55.993170 24402 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:38901
I20260620 08:10:55.993350 24531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:38901 every 8 connection(s)
I20260620 08:10:55.993592 24402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:10:55.996105 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 24402
I20260620 08:10:55.996209 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:10:55.996435 12819 external_mini_cluster.cc:1428] Setting key d80b9c3f4db6c57c3d5ed7ccb9d868e4
Jun 20 08:10:55 dist-test-slave-z0sp krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943055, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:10:56.006711 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.995847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:37429 (local address 127.12.132.254:43243)
0620 08:10:55.995911 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:55.995913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:55.996295 (+ 382us) server_negotiation.cc:407] Connection header received
0620 08:10:55.997109 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:55.997110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:55.997120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:55.997149 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:55.998109 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.999032 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.000553 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.000844 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.000883 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.003403 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:56.003404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:56.003404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:56.003411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:56.004907 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:56.005459 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:56.005460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:56.005461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:56.005535 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:56.005876 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:56.005877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:56.005877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:56.006052 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:56.006149 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.006521 (+ 372us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.006611 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 08:10:56.006946 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:55.995853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43243 (local address 127.12.132.195:37429)
0620 08:10:55.996162 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:55.996210 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:55.997002 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:55.997337 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:55.997341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:55.997699 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:10:55.997984 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:55.997990 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:55.999146 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:55.999147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.000409 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.000411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.000960 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.000961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.001067 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.001649 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:10:56.001660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:10:56.003282 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:10:56.005029 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:56.005037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:56.005039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:56.005365 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:56.005628 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:10:56.005629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:10:56.005630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:10:56.005786 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:10:56.006158 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:10:56.006165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:10:56.006340 (+ 175us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.006467 (+ 127us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.006709 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":93,"threads_started":1}
I20260620 08:10:56.007869 24532 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43243
I20260620 08:10:56.007980 24532 heartbeater.cc:464] Registering TS with master...
I20260620 08:10:56.008175 24532 heartbeater.cc:511] Master 127.12.132.254:43243 requested a full tablet report, sending...
I20260620 08:10:56.008602 24047 ts_manager.cc:194] Registered new tserver with Master: fb8e093df5134ff4900edfdaf092ccf3 (127.12.132.195:38901)
I20260620 08:10:56.009203 24047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:37429
I20260620 08:10:56.019409 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:10:56.028888 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.021126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.12.132.254:43243)
0620 08:10:56.021170 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.021172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.021348 (+ 176us) server_negotiation.cc:407] Connection header received
0620 08:10:56.021418 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.021419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.021440 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.021623 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:10:56.022369 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.023124 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.023917 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.024236 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.024327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.025669 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:10:56.025671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:10:56.025671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:10:56.025678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:10:56.027438 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:56.028009 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:56.028010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:56.028011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:56.028079 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:10:56.028281 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:10:56.028283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:10:56.028283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:10:56.028404 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:10:56.028532 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.028661 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.028750 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:10:56.030905 24047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:10:56.033403 24047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:10:56.044027 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:38365 (local address 127.0.0.1:33618)
0620 08:10:56.039131 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.039171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:56.039239 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:56.039535 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:56.039538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:56.039540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:56.039662 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.039665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.041270 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.041271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.042672 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.042674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.043706 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.043707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.043803 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.043829 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.043878 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.043937 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":15,"thread_start_us":108,"threads_started":1}
I20260620 08:10:56.044447 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:45289 (local address 127.0.0.1:44684)
0620 08:10:56.039656 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.039665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:56.039709 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:56.039951 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:56.039952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:56.039954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:56.039994 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.039995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.041397 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.041397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.042671 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.042674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.043788 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.043789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.043837 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.043840 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.043896 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.043937 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"mutex_wait_us":27,"thread_start_us":686,"threads_started":1}
I20260620 08:10:56.044557 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.12.132.194:38365)
0620 08:10:56.038927 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.038931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.039220 (+ 289us) server_negotiation.cc:407] Connection header received
0620 08:10:56.039378 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.039381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.039410 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.039743 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:56.039825 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.041177 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.042794 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.043569 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.044265 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.044328 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.044353 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.044395 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":165,"threads_started":1}
I20260620 08:10:56.045027 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.12.132.193:45289)
0620 08:10:56.038729 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.038734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.039767 (+ 1033us) server_negotiation.cc:407] Connection header received
0620 08:10:56.039807 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.039809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.039824 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.039893 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:56.040065 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.041304 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.042768 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.043636 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.043701 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.043758 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.043981 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.044021 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":85,"threads_started":1}
I20260620 08:10:56.045318 24331 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2b0d1684a347f2b95017ae5157ad78 (DEFAULT_TABLE table=test-table [id=07ba7dd44ccf49e28d416c592cb5e8ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:56.045706 24331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2b0d1684a347f2b95017ae5157ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:56.045770 24192 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2b0d1684a347f2b95017ae5157ad78 (DEFAULT_TABLE table=test-table [id=07ba7dd44ccf49e28d416c592cb5e8ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:56.046046 24192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2b0d1684a347f2b95017ae5157ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:56.046729 24546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.12.132.195:38901)
0620 08:10:56.038721 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.038727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.039552 (+ 825us) server_negotiation.cc:407] Connection header received
0620 08:10:56.039650 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.039652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.039669 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.039774 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:56.040203 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.042063 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.044282 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.045566 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.045702 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.045772 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.046513 (+ 741us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.046576 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":134,"threads_started":1}
I20260620 08:10:56.046119 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.038332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:38901 (local address 127.0.0.1:55708)
0620 08:10:56.039460 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.039469 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:56.039524 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:56.039935 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:56.039936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:56.039938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:56.040059 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.040061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.042182 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.042184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.044050 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.044053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.045853 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.045854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.045933 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.045936 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.045996 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.046043 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":2,"thread_start_us":566,"threads_started":1}
I20260620 08:10:56.049849 24550 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: Bootstrap starting.
I20260620 08:10:56.050319 24466 tablet_service.cc:1467] Processing CreateTablet for tablet 8d2b0d1684a347f2b95017ae5157ad78 (DEFAULT_TABLE table=test-table [id=07ba7dd44ccf49e28d416c592cb5e8ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:10:56.050626 24466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d2b0d1684a347f2b95017ae5157ad78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:10:56.052155 24550 tablet_bootstrap.cc:654] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:56.052901 24550 log.cc:824] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:56.054352 24550 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: No bootstrap required, opened a new log
I20260620 08:10:56.054450 24550 ts_tablet_manager.cc:1397] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 08:10:56.055478 24553 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: Bootstrap starting.
I20260620 08:10:56.055846 24552 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: Bootstrap starting.
I20260620 08:10:56.056342 24553 tablet_bootstrap.cc:654] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:56.056890 24553 log.cc:824] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:56.057401 24552 tablet_bootstrap.cc:654] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: Neither blocks nor log segments found. Creating new log.
I20260620 08:10:56.057965 24552 log.cc:824] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: Log is configured to *not* fsync() on all Append() calls
I20260620 08:10:56.058745 24552 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: No bootstrap required, opened a new log
I20260620 08:10:56.058835 24552 ts_tablet_manager.cc:1397] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:10:56.058717 24550 raft_consensus.cc:348] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.058882 24550 raft_consensus.cc:374] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:56.058917 24550 raft_consensus.cc:729] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f81f11222a9c4b42bde55070bff13a97, State: Initialized, Role: FOLLOWER
I20260620 08:10:56.059083 24550 consensus_queue.cc:260] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.059437 24550 ts_tablet_manager.cc:1428] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260620 08:10:56.059578 24397 heartbeater.cc:503] Master 127.12.132.254:43243 was elected leader, sending a full tablet report...
I20260620 08:10:56.061058 24553 tablet_bootstrap.cc:492] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: No bootstrap required, opened a new log
I20260620 08:10:56.061206 24553 ts_tablet_manager.cc:1397] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260620 08:10:56.062098 24552 raft_consensus.cc:348] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.062261 24552 raft_consensus.cc:374] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:56.062335 24552 raft_consensus.cc:729] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8e093df5134ff4900edfdaf092ccf3, State: Initialized, Role: FOLLOWER
I20260620 08:10:56.062458 24552 consensus_queue.cc:260] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.062836 24552 ts_tablet_manager.cc:1428] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260620 08:10:56.062879 24532 heartbeater.cc:503] Master 127.12.132.254:43243 was elected leader, sending a full tablet report...
I20260620 08:10:56.066336 24553 raft_consensus.cc:348] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.066538 24553 raft_consensus.cc:374] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:10:56.066589 24553 raft_consensus.cc:729] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35e237f6172647779e0dfe91d80616dc, State: Initialized, Role: FOLLOWER
I20260620 08:10:56.066740 24553 consensus_queue.cc:260] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.067229 24553 ts_tablet_manager.cc:1428] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260620 08:10:56.067396 24258 heartbeater.cc:503] Master 127.12.132.254:43243 was elected leader, sending a full tablet report...
I20260620 08:10:56.085644 24556 raft_consensus.cc:482] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:10:56.085844 24556 raft_consensus.cc:504] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.086467 24556 leader_election.cc:288] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35e237f6172647779e0dfe91d80616dc (127.12.132.193:45289), fb8e093df5134ff4900edfdaf092ccf3 (127.12.132.195:38901)
I20260620 08:10:56.099031 24399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.089194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:45289 (local address 127.12.132.194:52001)
0620 08:10:56.089276 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.089301 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:56.089408 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:56.095221 (+ 5813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:56.095224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:56.095227 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:56.095374 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.095377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.096691 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.096692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.097923 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.097927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.098757 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.098758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.098837 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.098841 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.098907 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.098954 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:10:56.099537 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.094676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:52001 (local address 127.12.132.193:45289)
0620 08:10:56.094793 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.094797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.094815 (+ 18us) server_negotiation.cc:407] Connection header received
0620 08:10:56.094856 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.094857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.094870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.094971 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:56.095499 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.096580 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.098028 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.098640 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.098714 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.098723 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.099430 (+ 707us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.099472 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 08:10:56.100253 24209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2b0d1684a347f2b95017ae5157ad78" candidate_uuid: "f81f11222a9c4b42bde55070bff13a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35e237f6172647779e0dfe91d80616dc" is_pre_election: true
I20260620 08:10:56.100457 24209 raft_consensus.cc:2454] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f81f11222a9c4b42bde55070bff13a97 in term 0.
I20260620 08:10:56.101013 24284 leader_election.cc:302] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35e237f6172647779e0dfe91d80616dc, f81f11222a9c4b42bde55070bff13a97; no voters:
I20260620 08:10:56.101239 24556 raft_consensus.cc:2781] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:10:56.101374 24556 raft_consensus.cc:482] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:10:56.101459 24556 raft_consensus.cc:3037] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:56.102471 24556 raft_consensus.cc:504] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.102826 24209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2b0d1684a347f2b95017ae5157ad78" candidate_uuid: "f81f11222a9c4b42bde55070bff13a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35e237f6172647779e0dfe91d80616dc"
I20260620 08:10:56.102905 24209 raft_consensus.cc:3037] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:56.103924 24209 raft_consensus.cc:2454] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f81f11222a9c4b42bde55070bff13a97 in term 1.
I20260620 08:10:56.104156 24284 leader_election.cc:302] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35e237f6172647779e0dfe91d80616dc, f81f11222a9c4b42bde55070bff13a97; no voters:
I20260620 08:10:56.104463 24556 leader_election.cc:288] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [CANDIDATE]: Term 1 election: Requested vote from peers 35e237f6172647779e0dfe91d80616dc (127.12.132.193:45289), fb8e093df5134ff4900edfdaf092ccf3 (127.12.132.195:38901)
I20260620 08:10:56.104578 24556 raft_consensus.cc:2781] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:10:56.104660 24556 raft_consensus.cc:686] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [term 1 LEADER]: Becoming Leader. State: Replica: f81f11222a9c4b42bde55070bff13a97, State: Running, Role: LEADER
I20260620 08:10:56.104787 24556 consensus_queue.cc:237] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } }
I20260620 08:10:56.105837 24047 catalog_manager.cc:5495] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 reported cstate change: term changed from 0 to 1, leader changed from <none> to f81f11222a9c4b42bde55070bff13a97 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "f81f11222a9c4b42bde55070bff13a97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f81f11222a9c4b42bde55070bff13a97" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 38365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 } health_report { overall_health: UNKNOWN } } }
I20260620 08:10:56.107740 24546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.096128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:51767 (local address 127.12.132.195:38901)
0620 08:10:56.096178 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.096180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.098902 (+ 2722us) server_negotiation.cc:407] Connection header received
0620 08:10:56.099054 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.099055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.099066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.099110 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:10:56.099418 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.100186 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.101581 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.107245 (+ 5664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.107303 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.107313 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.107605 (+ 292us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.107673 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 08:10:56.108098 24559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.095908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:38901 (local address 127.12.132.194:51767)
0620 08:10:56.098814 (+ 2906us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.098827 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:10:56.098945 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:10:56.099205 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:10:56.099206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:10:56.099208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:10:56.099304 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.099306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.100312 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.100312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.101472 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:10:56.101475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.107421 (+ 5946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.107422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:10:56.107511 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.107515 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:10:56.107949 (+ 434us) client_negotiation.cc:240] Negotiation successful
0620 08:10:56.108003 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2870,"thread_start_us":111,"threads_started":1}
I20260620 08:10:56.108757 24486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2b0d1684a347f2b95017ae5157ad78" candidate_uuid: "f81f11222a9c4b42bde55070bff13a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8e093df5134ff4900edfdaf092ccf3" is_pre_election: true
I20260620 08:10:56.108763 24485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d2b0d1684a347f2b95017ae5157ad78" candidate_uuid: "f81f11222a9c4b42bde55070bff13a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8e093df5134ff4900edfdaf092ccf3"
I20260620 08:10:56.108937 24486 raft_consensus.cc:2454] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f81f11222a9c4b42bde55070bff13a97 in term 0.
I20260620 08:10:56.117527 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.114045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.12.132.194:38365)
0620 08:10:56.114212 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 08:10:56.114214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:10:56.114220 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:10:56.114251 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:10:56.114252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:10:56.114270 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:10:56.114341 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:10:56.114521 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.115400 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.116560 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:10:56.116884 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:10:56.116983 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:10:56.117172 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:10:56.117301 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:10:56.117404 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 08:10:56.117428 (+ 24us) server_negotiation.cc:299] Negotiation successful
0620 08:10:56.117450 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260620 08:10:56.119285 24485 raft_consensus.cc:3037] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:10:56.119428 24209 raft_consensus.cc:1261] T 8d2b0d1684a347f2b95017ae5157ad78 P 35e237f6172647779e0dfe91d80616dc [term 1 FOLLOWER]: Refusing update from remote peer f81f11222a9c4b42bde55070bff13a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:10:56.119724 24560 consensus_queue.cc:1035] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35e237f6172647779e0dfe91d80616dc" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 45289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:10:56.120373 24485 raft_consensus.cc:1261] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 1 FOLLOWER]: Refusing update from remote peer f81f11222a9c4b42bde55070bff13a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:10:56.120760 24560 consensus_queue.cc:1035] T 8d2b0d1684a347f2b95017ae5157ad78 P f81f11222a9c4b42bde55070bff13a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb8e093df5134ff4900edfdaf092ccf3" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:10:56.126750 24563 mvcc.cc:205] Tried to move back new op lower bound from 7298838757862182912 to 7298838757806530560. Current Snapshot: MvccSnapshot[applied={T|T < 7298838757862182912}]
I20260620 08:10:56.128412 24566 mvcc.cc:205] Tried to move back new op lower bound from 7298838757862182912 to 7298838757806530560. Current Snapshot: MvccSnapshot[applied={T|T < 7298838757862182912}]
I20260620 08:10:56.131629 24564 mvcc.cc:205] Tried to move back new op lower bound from 7298838757862182912 to 7298838757806530560. Current Snapshot: MvccSnapshot[applied={T|T < 7298838757862182912}]
I20260620 08:10:56.133850 24048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:10:56.133955 24048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:10:56.134711 24048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b693e2ac71fe4211872c8581ca7998a5: Sending DeleteTablet for 3 replicas of tablet 8d2b0d1684a347f2b95017ae5157ad78
I20260620 08:10:56.135154 24331 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d2b0d1684a347f2b95017ae5157ad78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:10:56 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:33618
I20260620 08:10:56.135310 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 24129
I20260620 08:10:56.135576 24466 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d2b0d1684a347f2b95017ae5157ad78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:10:56 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:55708
W20260620 08:10:56.144116 24032 connection.cc:535] client connection to 127.12.132.193:45289 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 08:10:56.144213 24032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:10:56.144930 24574 tablet_replica.cc:330] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3: stopping tablet replica
I20260620 08:10:56.144927 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 24266
I20260620 08:10:56.145711 24574 raft_consensus.cc:2229] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:10:56.145900 24574 raft_consensus.cc:2258] T 8d2b0d1684a347f2b95017ae5157ad78 P fb8e093df5134ff4900edfdaf092ccf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:10:56.146365 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.146256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:45289 (local address 127.0.0.1:44700)
0620 08:10:56.146297 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.146334 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260620 08:10:56.146507 24032 catalog_manager.cc:4584] TS 35e237f6172647779e0dfe91d80616dc (127.12.132.193:45289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d2b0d1684a347f2b95017ae5157ad78: Network error: Client connection negotiation failed: client connection to 127.12.132.193:45289: connect: Connection refused (error 111)
I20260620 08:10:56.151964 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 24402
I20260620 08:10:56.152737 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.152643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:38365 (local address 127.0.0.1:33646)
0620 08:10:56.152684 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.152707 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:38365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 08:10:56.156486 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 24016
I20260620 08:10:56.156617 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:10:56.156476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:38901 (local address 127.0.0.1:55712)
0620 08:10:56.156556 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 08:10:56.156586 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.195:38901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260620 08:10:56.161271 12819 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 08:10:56.496205 12819 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 08:10:56.496321 12819 mini_ranger.cc:66] Stopping Ranger...
I20260620 08:10:56.841390 12819 mini_ranger.cc:68] Stopped Ranger
2026-06-20 08:10:56.841 UTC [23602] LOG: received smart shutdown request
2026-06-20 08:10:56.844 UTC [23602] LOG: background worker "logical replication launcher" (PID 23611) exited with exit code 1
2026-06-20 08:10:56.845 UTC [23606] LOG: shutting down
2026-06-20 08:10:57.050 UTC [23602] LOG: database system is shut down
2026-06-20T08:10:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50406 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:10:57 dist-test-slave-z0sp krb5kdc[24584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:10:57 dist-test-slave-z0sp krb5kdc[24584](info): set up 2 sockets
Jun 20 08:10:57 dist-test-slave-z0sp krb5kdc[24584](info): commencing operation
krb5kdc: starting...
W20260620 08:10:58.225072 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:10:58 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:10:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangeradmin_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangeradmin_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangerlookup_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangerlookup_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
I20260620 08:10:58.399559 12819 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZ6VfL9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/postgres -l logfile start
2026-06-20 08:11:01.333 UTC [24614] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 08:11:01.333 UTC [24614] LOG: listening on IPv4 address "127.12.132.212", port 46571
2026-06-20 08:11:01.337 UTC [24614] LOG: listening on Unix socket "/tmp/.s.PGSQL.46571"
2026-06-20 08:11:01.359 UTC [24617] LOG: database system was shut down at 2026-06-20 08:10:59 UTC
2026-06-20 08:11:01.363 UTC [24614] LOG: database system is ready to accept connections
I20260620 08:11:02.345515 12819 mini_postgres.cc:96] Postgres bound to 46571
2026-06-20 08:11:02.351 UTC [24625] FATAL: database "slave" does not exist
127.12.132.212:46571 - accepting connections
I20260620 08:11:02.352391 12819 mini_ranger.cc:161] Starting Ranger...
I20260620 08:11:02.367496 12819 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 08:11:02.960271 12819 mini_ranger.cc:87] Created ranger Postgres database
I20260620 08:11:02.960367 12819 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin
2026-06-20 08:11:03,243 [I] DB FLAVOR :POSTGRES
2026-06-20 08:11:03,243 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 08:11:03,243 [I] Checking connection..
2026-06-20 08:11:03,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 08:11:03,503 [I] Checking connection passed.
2026-06-20 08:11:03,504 [I] --------- Verifying version history table ---------
2026-06-20 08:11:03,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:11:03,748 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 08:11:03,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:11:03,992 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 08:11:03,992 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 08:11:03,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 08:11:04.253 UTC [24689] WARNING: there is no transaction in progress
2026-06-20 08:11:04,260 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 08:11:04,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 08:11:04,529 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 08:11:04,529 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 08:11:04,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 08:11:04,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 08:11:05,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z0sp.c.gcp-upstream.internal','N') ;"
2026-06-20 08:11:05,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 08:11:05,590 [I] Table x_portal_user does not exist in database ranger
2026-06-20 08:11:05,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 08:11:05,844 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 08:11:05,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 08:11:06,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 08:11:06,347 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 08:11:06,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 08:11:07.382 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.400 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.418 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.435 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.455 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.474 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.533 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.542 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.559 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.570 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.580 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.596 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.608 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.616 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.623 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:07.631 UTC [24797] WARNING: there is no transaction in progress
2026-06-20 08:11:08,384 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 08:11:08,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 08:11:08,638 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 08:11:08,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 08:11:08,913 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 08:11:08,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 08:11:09,180 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 08:11:09,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 08:11:09,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 08:11:09,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z0sp.c.gcp-upstream.internal';"
2026-06-20 08:11:09,950 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 08:11:09,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 08:11:10,239 [I] DB_PATCHES have already been applied
I20260620 08:11:10.245498 12819 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZ6VfL9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 08:11:10.245566 12819 mini_ranger.cc:193] Using host: 127.12.132.212
I20260620 08:11:10.249786 12819 mini_ranger.cc:227] Ranger admin URL: http://127.12.132.212:58333
Jun 20, 2026 8:11:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:11:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:11:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 8:11:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 8:11:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 8:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 8:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 08:11:11.374579 12819 mini_ranger.cc:160] Time spent starting Ranger: real 9.022s user 0.000s sys 0.011s
Jun 20, 2026 8:11:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 8:11:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 20, 2026 8:11:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 8:11:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 8:11:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 08:11:33.732666 12819 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.132.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.132.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangerkms_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.132.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/rangerkms_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.132.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.132.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.132.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/HTTP_127.12.132.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 08:11:33 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943093, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 08:11:33.837231 12819 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 08:11:33.847318 12819 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 08:11:34.851598 12819 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 08:11:34.851691 12819 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms
I20260620 08:11:34.851708 12819 mini_ranger_kms.cc:227] Using postgres at 127.12.132.212:46571
2026-06-20 08:11:35,064 [I] DB FLAVOR :POSTGRES
2026-06-20 08:11:35,064 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 08:11:35,065 [I] Checking connection
2026-06-20 08:11:35,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 08:11:35,315 [I] connection success
2026-06-20 08:11:35,315 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 08:11:35,315 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 08:11:35,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 08:11:35,601 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 08:11:35,601 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 08:11:35,601 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 08:11:35,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.132.212:46571/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 08:11:35,907 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 08:11:36.446625 12819 mini_ranger_kms.cc:310] Created kms service
I20260620 08:11:36.677063 12819 mini_ranger_kms.cc:326] Created kudu user
I20260620 08:11:36.734302 12819 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 08:11:36 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943093, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:37.190704 12819 mini_ranger_kms.cc:384] Added ranger policy
I20260620 08:11:37.190833 12819 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZ6VfL9/build/release/bin/postgresql.jar:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZ6VfL9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 08:11:37.190855 12819 mini_ranger_kms.cc:240] Using host: 127.12.132.212
I20260620 08:11:37.194743 12819 mini_ranger_kms.cc:276] Ranger KMS PID: 24978
I20260620 08:11:37.194819 12819 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.132.212:54043
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 8:11:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 08:11:39.512763 12819 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.676s user 0.007s sys 0.157s
I20260620 08:11:39.512861 12819 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 08:11:39.512907 12819 mini_ranger_kms.cc:401] 127.12.132.212:54043/kms/v1/keys
Jun 20 08:11:39 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781943099, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 08:11:40 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943099, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
Jun 20 08:11:40 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943099, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.132.212@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
Jun 20 08:11:42 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:11:42.861244 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37511
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37511
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:42.946081 25028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:42.946285 25028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:42.946309 25028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:42.947546 25028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:42.947592 25028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:42.947606 25028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:42.947619 25028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:42.947631 25028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:42.949278 25028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.132.212:54043/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37511
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:42.949492 25028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:42.949745 25028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:42.955533 25034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:42.955564 25036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:42.955576 25028 server_base.cc:1034] running on GCE node
W20260620 08:11:42.955523 25033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:42.955945 25028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:42.956259 25028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:42.957481 25028 hybrid_clock.cc:648] HybridClock initialized: now 1781943102957459 us; error 48 us; skew 500 ppm
Jun 20 08:11:42 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943102, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:42.963406 25028 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:42.963975 25028 webserver.cc:460] Webserver started at http://127.12.132.254:38545/ using document root <none> and password file <none>
I20260620 08:11:42.964226 25028 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:42.964289 25028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:42.964442 25028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:11:42 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943102, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:43.013212 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "864096ea78e14c17947649d78f82fee9"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "1ec1197485de681929a6760fa1c918ea"
server_key_iv: "117a730d915913af62d3f1bc5020c6ab"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:43.013630 25028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "864096ea78e14c17947649d78f82fee9"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "1ec1197485de681929a6760fa1c918ea"
server_key_iv: "117a730d915913af62d3f1bc5020c6ab"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:43.015070 25028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:43.027633 25043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:43.027863 25028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:43.027967 25028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "864096ea78e14c17947649d78f82fee9"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "1ec1197485de681929a6760fa1c918ea"
server_key_iv: "117a730d915913af62d3f1bc5020c6ab"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:43.028079 25028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:43.043761 25028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:43.046792 25028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:43.047026 25028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:43.053205 25028 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:37511
I20260620 08:11:43.053300 25095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:37511 every 8 connection(s)
I20260620 08:11:43.053666 25028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:43.054494 25096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:43.057647 25096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Bootstrap starting.
I20260620 08:11:43.057971 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25028
I20260620 08:11:43.058089 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:11:43.058277 12819 external_mini_cluster.cc:1428] Setting key 34eb335eaff44233038c5c258be332c0
I20260620 08:11:43.058367 25096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:43.058696 25096 log.cc:824] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:43.059801 25096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: No bootstrap required, opened a new log
I20260620 08:11:43.061754 25096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:43.061937 25096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:43.061970 25096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864096ea78e14c17947649d78f82fee9, State: Initialized, Role: FOLLOWER
I20260620 08:11:43.062137 25096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:43.062216 25096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:43.062244 25096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:43.062309 25096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:43.063011 25096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:43.063153 25096 leader_election.cc:302] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 864096ea78e14c17947649d78f82fee9; no voters:
I20260620 08:11:43.063315 25096 leader_election.cc:288] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:11:43.063357 25101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:43.063582 25101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 LEADER]: Becoming Leader. State: Replica: 864096ea78e14c17947649d78f82fee9, State: Running, Role: LEADER
I20260620 08:11:43.063664 25096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:43.063714 25101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:43.064880 25102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "864096ea78e14c17947649d78f82fee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } } }
I20260620 08:11:43.064971 25102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:43.065160 25096 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 08:11:43.065347 25103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 864096ea78e14c17947649d78f82fee9. Latest consensus state: current_term: 1 leader_uuid: "864096ea78e14c17947649d78f82fee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } } }
I20260620 08:11:43.065405 25103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: This master's current role is: LEADER
Jun 20 08:11:43 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:43.068825 25115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:43.069823 25115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:43.073639 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:43.058793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.12.132.254:37511)
0620 08:11:43.059130 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:43.059140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:43.059174 (+ 34us) server_negotiation.cc:407] Connection header received
0620 08:11:43.059977 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:43.059988 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:43.060057 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:43.060216 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:43.060941 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:43.062335 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:43.063085 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:43.063763 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:43.063867 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:43.068550 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:43.068586 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:43.068587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:43.068692 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:43.071772 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:43.072338 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:43.072341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:43.072341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:43.072442 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:43.072671 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:43.072672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:43.072672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:43.072994 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:43.073152 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:43.073266 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 08:11:43.073425 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":75,"threads_started":1}
I20260620 08:11:43.074920 25115 catalog_manager.cc:1347] Generated new cluster ID: 6b0e6cbe26924eca8cd33a3887063494
I20260620 08:11:43.074972 25115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:11:43.110828 25115 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:11:43.111678 25115 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:43.121508 25115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Generated new TSK 0
I20260620 08:11:43.121701 25115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:11:43.858384 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:43.969570 25141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:43.969847 25141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:43.969883 25141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:43.971586 25141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:43.971700 25141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:43.971760 25141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:11:43.974105 25141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:43.974344 25141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:43.974627 25141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:43.981268 25146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:43.981596 25141 server_base.cc:1034] running on GCE node
W20260620 08:11:43.981412 25147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:43.981293 25149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:43.982020 25141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:43.982317 25141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:43.983497 25141 hybrid_clock.cc:648] HybridClock initialized: now 1781943103983487 us; error 41 us; skew 500 ppm
Jun 20 08:11:43 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:43.986805 25141 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:11:43.987692 25141 webserver.cc:460] Webserver started at http://127.12.132.193:36515/ using document root <none> and password file <none>
I20260620 08:11:43.988008 25141 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:43.988065 25141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:43.988263 25141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:11:43 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:44.003224 25141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "de5513b71f7c4a06a7720ea4f19c3db3"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "3f955c3697952cc1a212b21896a97285"
server_key_iv: "c099424f075b8b4d51780e97d9462be9"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.003715 25141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "de5513b71f7c4a06a7720ea4f19c3db3"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "3f955c3697952cc1a212b21896a97285"
server_key_iv: "c099424f075b8b4d51780e97d9462be9"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.006805 25141 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 08:11:44.024804 25158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.025092 25141 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260620 08:11:44.025202 25141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "de5513b71f7c4a06a7720ea4f19c3db3"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "3f955c3697952cc1a212b21896a97285"
server_key_iv: "c099424f075b8b4d51780e97d9462be9"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.025305 25141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:44.077643 25141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:44.080534 25141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:44.080788 25141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:44.081084 25141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:44.081533 25141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:44.081621 25141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.081720 25141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:44.081792 25141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.091316 25141 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:46469
I20260620 08:11:44.091812 25141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:11:44.092142 25271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:46469 every 8 connection(s)
I20260620 08:11:44.094874 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25141
I20260620 08:11:44.094976 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:11:44.095167 12819 external_mini_cluster.cc:1428] Setting key 15bf761cbdbf06eb88389832bc8358af
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:44.111007 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.092736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.193:39495)
0620 08:11:44.100060 (+ 7324us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:44.100124 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:44.100907 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:44.101278 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:44.101283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:44.101551 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:44.101846 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.101852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.102937 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.102938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.104146 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.104149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.104714 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.104715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.104851 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.105470 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:44.105480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:44.107178 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:44.109270 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.109276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.109278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.109585 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.109831 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.109832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.109833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.109968 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.110288 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:44.110294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:44.110470 (+ 176us) client_negotiation.cc:769] Sending connection context
0620 08:11:44.110583 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 08:11:44.110763 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7261,"thread_start_us":78,"threads_started":1}
I20260620 08:11:44.111685 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.092743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:39495 (local address 127.12.132.254:37511)
0620 08:11:44.092895 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:44.092897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:44.100556 (+ 7659us) server_negotiation.cc:407] Connection header received
0620 08:11:44.101077 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:44.101078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:44.101093 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:44.101153 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:44.101975 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.102831 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.104241 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.104570 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.104631 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.107286 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:44.107288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:44.107288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:44.107296 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:44.109161 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.109684 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.109685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.109686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.109753 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.110053 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.110054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.110054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.110199 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:44.110261 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:44.111490 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0620 08:11:44.111582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260620 08:11:44.112434 25272 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:44.112550 25272 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:44.112747 25272 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:44.113348 25060 ts_manager.cc:194] Registered new tserver with Master: de5513b71f7c4a06a7720ea4f19c3db3 (127.12.132.193:46469)
I20260620 08:11:44.114228 25060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:39495
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:11:44.156950 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 08:11:44.291404 25280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:44.291970 25280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:44.292157 25280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:44.295074 25280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:44.295377 25280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:44.295495 25280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:11:44.298352 25280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:44.298803 25280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:44.299268 25280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:44.306491 25286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:44.306866 25288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:44.307688 25280 server_base.cc:1034] running on GCE node
W20260620 08:11:44.308034 25285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:44.308316 25280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:44.308683 25280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:44.309885 25280 hybrid_clock.cc:648] HybridClock initialized: now 1781943104309855 us; error 38 us; skew 500 ppm
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:44.312474 25280 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:11:44.312961 25280 webserver.cc:460] Webserver started at http://127.12.132.194:41141/ using document root <none> and password file <none>
I20260620 08:11:44.313215 25280 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:44.313284 25280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:44.313426 25280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:44.326370 25280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "f7d1512f8a92bb055c59fafb8cacef21"
server_key_iv: "5bc143222a1cb129d640778229e8e183"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.327606 25280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "f7d1512f8a92bb055c59fafb8cacef21"
server_key_iv: "5bc143222a1cb129d640778229e8e183"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.329643 25280 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:44.350026 25295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.350289 25280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:44.350389 25280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "f7d1512f8a92bb055c59fafb8cacef21"
server_key_iv: "5bc143222a1cb129d640778229e8e183"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.350510 25280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:44.390944 25280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:44.393594 25280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:44.393833 25280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:44.394179 25280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:44.394603 25280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:44.394662 25280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.394704 25280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:44.394731 25280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.403347 25280 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:33867
I20260620 08:11:44.403463 25409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:33867 every 8 connection(s)
I20260620 08:11:44.403769 25280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:11:44.406677 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25280
I20260620 08:11:44.406783 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:11:44.406968 12819 external_mini_cluster.cc:1428] Setting key ddfb7b05a0b8912f7673d0d1a686c50b
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:44.418006 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.405414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:38319 (local address 127.12.132.254:37511)
0620 08:11:44.405493 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:44.405496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:44.405644 (+ 148us) server_negotiation.cc:407] Connection header received
0620 08:11:44.406214 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:44.406215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:44.406232 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:44.406329 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:44.408742 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.409515 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.411052 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.411380 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.411472 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.414489 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:44.414490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:44.414491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:44.414498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:44.416227 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.416761 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.416762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.416763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.416826 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.417133 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.417134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.417135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.417291 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:44.417351 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:44.417767 (+ 416us) server_negotiation.cc:299] Negotiation successful
0620 08:11:44.417867 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 08:11:44.418915 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.405313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.194:38319)
0620 08:11:44.405506 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:44.405549 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:44.406111 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:44.406443 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:44.406448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:44.407447 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:44.408626 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.408633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.409641 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.409642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.410946 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.410948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.411518 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.411520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.411665 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.412644 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:44.412654 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:44.414354 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:44.416322 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.416328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.416330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.416661 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.416905 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.416906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.416906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.417048 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.417390 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:44.417396 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:44.417589 (+ 193us) client_negotiation.cc:769] Sending connection context
0620 08:11:44.418378 (+ 789us) client_negotiation.cc:240] Negotiation successful
0620 08:11:44.418652 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260620 08:11:44.419826 25410 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:44.419926 25410 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:44.420130 25410 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:44.420619 25060 ts_manager.cc:194] Registered new tserver with Master: 8d44edc0b8a845bb81c8d97d0c1b1d1f (127.12.132.194:33867)
I20260620 08:11:44.421097 25060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:38319
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:11:44.448877 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:44.557113 25415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:44.557734 25415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:44.557904 25415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:44.560715 25415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:44.560873 25415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:44.560976 25415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:11:44.563244 25415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:44.563812 25415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:44.564121 25415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:44.571414 25420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:44.571555 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:44.571396 25421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:44.572774 25415 server_base.cc:1034] running on GCE node
I20260620 08:11:44.573006 25415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:44.573287 25415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:44.574473 25415 hybrid_clock.cc:648] HybridClock initialized: now 1781943104574451 us; error 38 us; skew 500 ppm
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:44.577677 25415 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:11:44.578186 25415 webserver.cc:460] Webserver started at http://127.12.132.195:43461/ using document root <none> and password file <none>
I20260620 08:11:44.578490 25415 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:44.578553 25415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:44.578714 25415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:44.593189 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "7ad21d6417ed4ea9834e6e0ef96b9743"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "a657e37b790e738ee5e345bc60d208ee"
server_key_iv: "2261ba0de422f6b5f9768c5e184f90c4"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.593653 25415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "7ad21d6417ed4ea9834e6e0ef96b9743"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "a657e37b790e738ee5e345bc60d208ee"
server_key_iv: "2261ba0de422f6b5f9768c5e184f90c4"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.596983 25415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260620 08:11:44.617340 25430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.617601 25415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:44.617692 25415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "7ad21d6417ed4ea9834e6e0ef96b9743"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "a657e37b790e738ee5e345bc60d208ee"
server_key_iv: "2261ba0de422f6b5f9768c5e184f90c4"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.617790 25415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:44.646030 25415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:44.648550 25415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:44.648773 25415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:44.649055 25415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:44.649475 25415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:44.649556 25415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.649626 25415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:44.649679 25415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.659032 25415 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:33191
I20260620 08:11:44.659574 25415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:11:44.660048 25543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:33191 every 8 connection(s)
I20260620 08:11:44.666608 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25415
I20260620 08:11:44.666723 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:11:44.666924 12819 external_mini_cluster.cc:1428] Setting key 8c7dc951532459a4cfc96f964af822c4
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:44.673408 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.661252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:53287 (local address 127.12.132.254:37511)
0620 08:11:44.661300 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:44.661302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:44.661584 (+ 282us) server_negotiation.cc:407] Connection header received
0620 08:11:44.662300 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:44.662302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:44.662314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:44.662346 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:44.663168 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.664169 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.666110 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.666495 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.666570 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.670097 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:44.670099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:44.670099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:44.670107 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:44.671833 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.672306 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.672307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.672308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.672371 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:44.672663 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:44.672664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:44.672664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:44.672806 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:44.672848 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:44.673233 (+ 385us) server_negotiation.cc:299] Negotiation successful
0620 08:11:44.673318 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:11:44.674175 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:44.660982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.195:53287)
0620 08:11:44.661459 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:44.661503 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:44.662173 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:44.662525 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:44.662529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:44.662772 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:44.663058 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.663064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.664424 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.664425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.665957 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:44.665963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:44.666852 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:44.666854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:44.667074 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:44.668230 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:44.668241 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:44.669983 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:44.671932 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.671938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.671940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.672216 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.672439 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:44.672440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:44.672440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:44.672585 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:44.672898 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:44.672904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:44.673076 (+ 172us) client_negotiation.cc:769] Sending connection context
0620 08:11:44.673753 (+ 677us) client_negotiation.cc:240] Negotiation successful
0620 08:11:44.673940 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":80,"threads_started":1}
I20260620 08:11:44.674829 25544 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:44.674940 25544 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:44.675138 25544 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:44.675544 25060 ts_manager.cc:194] Registered new tserver with Master: 7ad21d6417ed4ea9834e6e0ef96b9743 (127.12.132.195:33191)
I20260620 08:11:44.676049 25060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:53287
I20260620 08:11:44.678841 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:11:44.679548 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25141
I20260620 08:11:44.687283 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25280
I20260620 08:11:44.699719 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25415
I20260620 08:11:44.706959 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25028
I20260620 08:11:44.716146 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37511
--webserver_interface=127.12.132.254
--webserver_port=38545
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:37511
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:44.798974 25547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:44.799268 25547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:44.799324 25547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:44.800448 25547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:44.800530 25547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:44.800556 25547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:44.800590 25547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:44.800619 25547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:44.802156 25547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:37511
--ranger_config_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=38545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:44.802390 25547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:44.802624 25547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:44.808326 25552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:44.808411 25555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:44.808679 25547 server_base.cc:1034] running on GCE node
W20260620 08:11:44.808336 25553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:44.808975 25547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:44.809295 25547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:44.810490 25547 hybrid_clock.cc:648] HybridClock initialized: now 1781943104810476 us; error 31 us; skew 500 ppm
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:44.813419 25547 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:44.814116 25547 webserver.cc:460] Webserver started at http://127.12.132.254:38545/ using document root <none> and password file <none>
I20260620 08:11:44.814383 25547 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:44.814469 25547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 08:11:44 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:44.860941 25547 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:44.862272 25563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:44.862550 25547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:44.862663 25547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "864096ea78e14c17947649d78f82fee9"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "1ec1197485de681929a6760fa1c918ea"
server_key_iv: "117a730d915913af62d3f1bc5020c6ab"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:44.863140 25547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:44.888983 25547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:44.891594 25547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:44.891755 25547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:44.896803 25547 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:37511
I20260620 08:11:44.896832 25615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:37511 every 8 connection(s)
I20260620 08:11:44.897286 25547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:44.898579 25616 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:11:44.899250 25616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Bootstrap starting.
I20260620 08:11:44.903366 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25547
I20260620 08:11:44.903731 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:46469
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:37511
--webserver_port=36515
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:44.908775 25616 log.cc:824] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:44.910444 25616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:11:44.910760 25616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Bootstrap complete.
I20260620 08:11:44.913335 25616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:44.913587 25616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864096ea78e14c17947649d78f82fee9, State: Initialized, Role: FOLLOWER
I20260620 08:11:44.913779 25616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:44.913887 25616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:44.913955 25616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:44.914011 25616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:11:44.915015 25616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:44.915134 25616 leader_election.cc:302] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 864096ea78e14c17947649d78f82fee9; no voters:
I20260620 08:11:44.915300 25616 leader_election.cc:288] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 08:11:44.915369 25620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:11:44.915526 25620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [term 2 LEADER]: Becoming Leader. State: Replica: 864096ea78e14c17947649d78f82fee9, State: Running, Role: LEADER
I20260620 08:11:44.915670 25620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } }
I20260620 08:11:44.915657 25616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:44.915934 25622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "864096ea78e14c17947649d78f82fee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } } }
I20260620 08:11:44.915992 25622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:44.916201 25621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 864096ea78e14c17947649d78f82fee9. Latest consensus state: current_term: 2 leader_uuid: "864096ea78e14c17947649d78f82fee9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864096ea78e14c17947649d78f82fee9" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 37511 } } }
I20260620 08:11:44.916306 25621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:44.916474 25624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:44.917028 25624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:44.917222 25624 catalog_manager.cc:1259] Loaded cluster ID: 6b0e6cbe26924eca8cd33a3887063494
I20260620 08:11:44.917307 25624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:11:44.917383 25616 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 08:11:44.922044 25624 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:44.922765 25624 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Loaded TSK: 0
I20260620 08:11:44.923100 25624 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:11:45.023933 25618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:45.024319 25618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:45.024426 25618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:45.026237 25618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:45.026459 25618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:45.026558 25618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:11:45.028820 25618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:46469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=36515
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:45.029290 25618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:45.029611 25618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:45.036550 25654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.036909 25653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.037163 25656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:45.038198 25618 server_base.cc:1034] running on GCE node
I20260620 08:11:45.038527 25618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:45.038781 25618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:45.039983 25618 hybrid_clock.cc:648] HybridClock initialized: now 1781943105039949 us; error 45 us; skew 500 ppm
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:45.043272 25618 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:11:45.043877 25618 webserver.cc:460] Webserver started at http://127.12.132.193:36515/ using document root <none> and password file <none>
I20260620 08:11:45.044121 25618 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:45.044189 25618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:45.067261 25618 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260620 08:11:45.068468 25664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.068735 25618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:45.068818 25618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "de5513b71f7c4a06a7720ea4f19c3db3"
format_stamp: "Formatted at 2026-06-20 08:11:43 on dist-test-slave-z0sp"
server_key: "3f955c3697952cc1a212b21896a97285"
server_key_iv: "c099424f075b8b4d51780e97d9462be9"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:45.069293 25618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:45.092618 25618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:45.095028 25618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:45.095224 25618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:45.095505 25618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:45.095876 25618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:45.095945 25618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.095997 25618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:45.096032 25618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.103367 25618 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:46469
I20260620 08:11:45.103909 25618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:11:45.104609 25777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:46469 every 8 connection(s)
I20260620 08:11:45.107080 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25618
I20260620 08:11:45.107503 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:33867
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:37511
--webserver_port=41141
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:45.121914 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.105256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.193:33161)
0620 08:11:45.105443 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:45.105484 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:45.105971 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:45.106870 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:45.106875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:45.107183 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:45.107546 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.107554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.111951 (+ 4397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.111953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.113481 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.113486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.114405 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.114406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.114534 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.115148 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:45.115160 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:45.117028 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:45.119684 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.119691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.119694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.120018 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.120359 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.120360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.120361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.120581 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.121144 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:45.121150 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:45.121330 (+ 180us) client_negotiation.cc:769] Sending connection context
0620 08:11:45.121443 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 08:11:45.121620 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260620 08:11:45.122802 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.105322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:33161 (local address 127.12.132.254:37511)
0620 08:11:45.106137 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:45.106144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:45.106257 (+ 113us) server_negotiation.cc:407] Connection header received
0620 08:11:45.106479 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:45.106487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:45.106545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:45.109897 (+ 3352us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:45.110054 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.111748 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.113624 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.114260 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.114417 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.117198 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:45.117203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:45.117203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:45.117215 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:45.119578 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.120121 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.120124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.120125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.120260 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.120670 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.120671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.120672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.121021 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:45.121112 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:45.122213 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0620 08:11:45.122340 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":90,"threads_started":1}
I20260620 08:11:45.123539 25778 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:45.123641 25778 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.123865 25778 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.124822 25778 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37511 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 08:11:45.124912 25778 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.124977 25778 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:45.125334 25778 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.125407 25778 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.125707 25778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:11:45.210614 25782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:45.210877 25782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:45.210919 25782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:45.212514 25782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:45.212579 25782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:45.212625 25782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:11:45.214735 25782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=41141
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:45.215147 25782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:45.215440 25782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:45.222077 25790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.222625 25788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.222664 25787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:45.224408 25782 server_base.cc:1034] running on GCE node
I20260620 08:11:45.224748 25782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:45.225109 25782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:45.228421 25782 hybrid_clock.cc:648] HybridClock initialized: now 1781943105228401 us; error 39 us; skew 500 ppm
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:45.232359 25782 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:11:45.233035 25782 webserver.cc:460] Webserver started at http://127.12.132.194:41141/ using document root <none> and password file <none>
I20260620 08:11:45.233392 25782 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:45.233547 25782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:45.258498 25782 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.001s
I20260620 08:11:45.260936 25798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.261123 25782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260620 08:11:45.261217 25782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "f7d1512f8a92bb055c59fafb8cacef21"
server_key_iv: "5bc143222a1cb129d640778229e8e183"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:45.261719 25782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:45.287837 25782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:45.290787 25782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:45.291066 25782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:45.291436 25782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:45.291978 25782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:45.292109 25782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.292196 25782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:45.292258 25782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.300639 25782 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:33867
I20260620 08:11:45.300734 25911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:33867 every 8 connection(s)
I20260620 08:11:45.301268 25782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:11:45.301945 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25782
I20260620 08:11:45.302312 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.132.212:54043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:33191
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:37511
--webserver_port=43461
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:45.318073 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.305703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:47739 (local address 127.12.132.254:37511)
0620 08:11:45.305772 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:45.305775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:45.305960 (+ 185us) server_negotiation.cc:407] Connection header received
0620 08:11:45.306613 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:45.306615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:45.306629 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:45.306708 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:45.307443 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.308270 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.309688 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.310015 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.310105 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.312943 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:45.312944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:45.312945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:45.312953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:45.316056 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.316801 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.316803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.316803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.316866 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.317246 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.317247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.317247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.317396 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:45.317475 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:45.317865 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 08:11:45.317969 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 08:11:45.318514 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.305403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.194:47739)
0620 08:11:45.305802 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:45.305848 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:45.306481 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:45.306792 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:45.306797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:45.307049 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:45.307327 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.307332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.308387 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.308388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.309582 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.309585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.310161 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.310163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.310304 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.310960 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:45.310980 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:45.312803 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:45.316165 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.316173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.316176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.316596 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.316983 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.316984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.316984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.317144 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.317480 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:45.317487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:45.317678 (+ 191us) client_negotiation.cc:769] Sending connection context
0620 08:11:45.317803 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 08:11:45.317997 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":101,"threads_started":1}
I20260620 08:11:45.319010 25912 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:45.319121 25912 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.319330 25912 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.320371 25912 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37511 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 08:11:45.320449 25912 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.320538 25912 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:45.320928 25912 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.320998 25912 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.321280 25912 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:11:45.407320 25914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:45.407658 25914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:45.407732 25914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:45.409436 25914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:45.409535 25914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:45.409617 25914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:11:45.411784 25914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.132.212:54043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:33191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=43461
--tserver_master_addrs=127.12.132.254:37511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:45.412195 25914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:45.412519 25914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:45.419005 25921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.418998 25920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:45.419415 25923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:45.419988 25914 server_base.cc:1034] running on GCE node
I20260620 08:11:45.420205 25914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:45.420454 25914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:45.421654 25914 hybrid_clock.cc:648] HybridClock initialized: now 1781943105421635 us; error 33 us; skew 500 ppm
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:45.424872 25914 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:11:45.425381 25914 webserver.cc:460] Webserver started at http://127.12.132.195:43461/ using document root <none> and password file <none>
I20260620 08:11:45.425649 25914 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:45.425724 25914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:45.444983 25914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:45.450644 25931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.450850 25914 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260620 08:11:45.450939 25914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "7ad21d6417ed4ea9834e6e0ef96b9743"
format_stamp: "Formatted at 2026-06-20 08:11:44 on dist-test-slave-z0sp"
server_key: "a657e37b790e738ee5e345bc60d208ee"
server_key_iv: "2261ba0de422f6b5f9768c5e184f90c4"
server_key_version: "kuduclusterkey@0"
I20260620 08:11:45.451416 25914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:45.474601 25914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:45.476974 25914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:45.477180 25914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:45.477420 25914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:45.477787 25914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:45.477856 25914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.477914 25914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:45.477957 25914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:45.486186 25914 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:33191
I20260620 08:11:45.487099 25914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:11:45.489673 26045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:33191 every 8 connection(s)
I20260620 08:11:45.493191 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 25914
Jun 20 08:11:45 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943105, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:45.504879 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.493639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40244 (local address 127.12.132.254:37511)
0620 08:11:45.493744 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:45.493745 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:45.493854 (+ 109us) server_negotiation.cc:407] Connection header received
0620 08:11:45.493953 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:45.493954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:45.493966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:45.494017 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:45.494783 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.495484 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.496982 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.497316 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.497363 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.499630 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:45.499632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:45.499633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:45.499642 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:45.502944 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.503630 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.503631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.503632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.503726 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.503997 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.503998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.503998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.504153 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:45.504315 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:45.504595 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 08:11:45.504730 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":51,"threads_started":1}
I20260620 08:11:45.508873 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.492120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:35587 (local address 127.12.132.254:37511)
0620 08:11:45.492172 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:45.492175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:45.492467 (+ 292us) server_negotiation.cc:407] Connection header received
0620 08:11:45.493491 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:45.493492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:45.493506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:45.493544 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:45.494583 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.495412 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.497254 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.497516 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.497566 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.501105 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:45.501107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:45.501108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:45.501116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:45.503570 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.504756 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.504757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.504757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.504815 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:45.505112 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:45.505113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:45.505114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:45.505233 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:45.505286 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:45.508684 (+ 3398us) server_negotiation.cc:299] Negotiation successful
0620 08:11:45.508772 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":22}
I20260620 08:11:45.509742 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:45.492138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:37511 (local address 127.12.132.195:35587)
0620 08:11:45.492325 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:45.492365 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:45.492927 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:45.493896 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:45.493900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:45.494157 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:45.494470 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.494477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.495519 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.495520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.497152 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:45.497157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:45.497695 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:45.497696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:45.497880 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:45.498464 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:45.498478 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:45.500683 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:45.503677 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.503689 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.503692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.504664 (+ 972us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.504883 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:45.504884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:45.504884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:45.505039 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:45.506567 (+ 1528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:45.506574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:45.508455 (+ 1881us) client_negotiation.cc:769] Sending connection context
0620 08:11:45.508644 (+ 189us) client_negotiation.cc:240] Negotiation successful
0620 08:11:45.509558 (+ 914us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
I20260620 08:11:45.510742 26046 heartbeater.cc:347] Connected to a master server at 127.12.132.254:37511
I20260620 08:11:45.510852 26046 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.511045 26046 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.511667 26046 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:37511 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 08:11:45.511739 26046 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.511799 26046 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:45.512130 26046 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:45.512204 26046 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
W20260620 08:11:45.512517 26046 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 08:11:46 dist-test-slave-z0sp krb5kdc[24584](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781943106, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:46.125893 25778 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:46.126080 25778 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:46.126432 25580 ts_manager.cc:194] Registered new tserver with Master: de5513b71f7c4a06a7720ea4f19c3db3 (127.12.132.193:46469)
I20260620 08:11:46.127156 25580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:33161
I20260620 08:11:46.321458 25912 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:46.321614 25912 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:46.322033 25580 ts_manager.cc:194] Registered new tserver with Master: 8d44edc0b8a845bb81c8d97d0c1b1d1f (127.12.132.194:33867)
I20260620 08:11:46.322549 25580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:47739
Jun 20 08:11:46 dist-test-slave-z0sp krb5kdc[24584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943106, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for HTTP/127.12.132.212@KRBTEST.COM
I20260620 08:11:46.512676 26046 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:46.512848 26046 heartbeater.cc:511] Master 127.12.132.254:37511 requested a full tablet report, sending...
I20260620 08:11:46.513170 25580 ts_manager.cc:194] Registered new tserver with Master: 7ad21d6417ed4ea9834e6e0ef96b9743 (127.12.132.195:33191)
I20260620 08:11:46.513689 25580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:35587
I20260620 08:11:46.516538 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:11:46.525125 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.518023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.12.132.254:37511)
0620 08:11:46.518284 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.518287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.518448 (+ 161us) server_negotiation.cc:407] Connection header received
0620 08:11:46.518472 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.518473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.518486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.518554 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:46.519111 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.519609 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.520361 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.520660 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.520721 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.522398 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:46.522400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:46.522400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:46.522408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:46.523940 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:46.524289 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:46.524291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:46.524291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:46.524352 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:46.524583 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:46.524584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:46.524584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:46.524700 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:46.524825 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.524914 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.524998 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":59,"threads_started":1}
I20260620 08:11:46.527422 25580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:11:46.529273 25580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:11:46.553819 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:46469 (local address 127.0.0.1:34166)
0620 08:11:46.545743 (+ 5071us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.545753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:46.545789 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:46.546127 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:46.546128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:46.546130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:46.546170 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.546172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.550337 (+ 4165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.550338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.551777 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.551780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.553488 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.553489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.553575 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.553590 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 08:11:46.553665 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 08:11:46.553721 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5056,"thread_start_us":73,"threads_started":1}
I20260620 08:11:46.554618 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33191 (local address 127.0.0.1:44676)
0620 08:11:46.544644 (+ 4219us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.544694 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:46.544764 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:46.545538 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:46.545541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:46.545544 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:46.545638 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.545640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.547438 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.547439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.549090 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.549093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.554362 (+ 5269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.554363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.554455 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.554458 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:11:46.554506 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 08:11:46.554547 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4177,"thread_start_us":84,"threads_started":1}
I20260620 08:11:46.555063 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.12.132.195:33191)
0620 08:11:46.545005 (+ 4328us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.545011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.545021 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:11:46.545057 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.545059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.545075 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.545140 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:46.546247 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.547327 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.551868 (+ 4541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.554212 (+ 2344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.554282 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.554329 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.554786 (+ 457us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.554985 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4294,"thread_start_us":40,"threads_started":1}
I20260620 08:11:46.555218 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.12.132.193:46469)
0620 08:11:46.545898 (+ 5118us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.545902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.545912 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:11:46.545947 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.545949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.545964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.546041 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:46.549194 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.550236 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.552545 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.553304 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.553382 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.553450 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.554867 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.554913 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5080,"thread_start_us":52,"threads_started":1}
I20260620 08:11:46.555759 25980 tablet_service.cc:1467] Processing CreateTablet for tablet e2d7466f780448c4bb90f122eecdc1af (DEFAULT_TABLE table=test-table [id=30444d4c565845b0bc00678f098b17ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:46.555878 25712 tablet_service.cc:1467] Processing CreateTablet for tablet e2d7466f780448c4bb90f122eecdc1af (DEFAULT_TABLE table=test-table [id=30444d4c565845b0bc00678f098b17ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:46.556119 25980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d7466f780448c4bb90f122eecdc1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:46.556233 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d7466f780448c4bb90f122eecdc1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:46.559454 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:33867 (local address 127.0.0.1:60040)
0620 08:11:46.544868 (+ 4432us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.544876 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:46.544909 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:46.555820 (+ 10911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:46.555821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:46.555823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:46.555864 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.555866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.557195 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.557196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.558273 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.558277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.559220 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.559221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.559288 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.559293 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:11:46.559326 (+ 33us) client_negotiation.cc:240] Negotiation successful
0620 08:11:46.559366 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4417,"thread_start_us":124,"threads_started":1}
I20260620 08:11:46.559841 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.540869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.12.132.194:33867)
0620 08:11:46.555364 (+ 14495us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.555368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.555379 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:11:46.555430 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.555432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.555448 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.555980 (+ 532us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:46.556091 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.557091 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.558378 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.559109 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.559628 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.559687 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.559713 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.559763 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14450,"thread_start_us":44,"threads_started":1}
I20260620 08:11:46.560817 25846 tablet_service.cc:1467] Processing CreateTablet for tablet e2d7466f780448c4bb90f122eecdc1af (DEFAULT_TABLE table=test-table [id=30444d4c565845b0bc00678f098b17ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:46.561409 25846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d7466f780448c4bb90f122eecdc1af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:46.564662 26079 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: Bootstrap starting.
I20260620 08:11:46.565379 26080 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Bootstrap starting.
I20260620 08:11:46.565529 26079 tablet_bootstrap.cc:654] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:46.565922 26079 log.cc:824] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:46.566507 26080 tablet_bootstrap.cc:654] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:46.566694 26079 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: No bootstrap required, opened a new log
I20260620 08:11:46.566751 26079 ts_tablet_manager.cc:1397] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:46.567467 26080 log.cc:824] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:46.569296 26079 raft_consensus.cc:348] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.569464 26079 raft_consensus.cc:374] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:46.569495 26079 raft_consensus.cc:729] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5513b71f7c4a06a7720ea4f19c3db3, State: Initialized, Role: FOLLOWER
I20260620 08:11:46.569655 26079 consensus_queue.cc:260] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.569878 26079 ts_tablet_manager.cc:1428] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:11:46.570235 25778 heartbeater.cc:503] Master 127.12.132.254:37511 was elected leader, sending a full tablet report...
I20260620 08:11:46.571233 26085 raft_consensus.cc:482] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:46.571556 26085 raft_consensus.cc:504] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.572654 26085 leader_election.cc:288] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d44edc0b8a845bb81c8d97d0c1b1d1f (127.12.132.194:33867), 7ad21d6417ed4ea9834e6e0ef96b9743 (127.12.132.195:33191)
I20260620 08:11:46.572911 26081 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Bootstrap starting.
I20260620 08:11:46.573571 26081 tablet_bootstrap.cc:654] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:46.573863 26081 log.cc:824] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:46.573992 26080 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: No bootstrap required, opened a new log
I20260620 08:11:46.574097 26080 ts_tablet_manager.cc:1397] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260620 08:11:46.576287 26080 raft_consensus.cc:348] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.576525 26080 raft_consensus.cc:374] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:46.576617 26080 raft_consensus.cc:729] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad21d6417ed4ea9834e6e0ef96b9743, State: Initialized, Role: FOLLOWER
I20260620 08:11:46.576819 26080 consensus_queue.cc:260] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.577129 26080 ts_tablet_manager.cc:1428] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:11:46.577291 26046 heartbeater.cc:503] Master 127.12.132.254:37511 was elected leader, sending a full tablet report...
I20260620 08:11:46.583477 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.572237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:35377 (local address 127.12.132.194:33867)
0620 08:11:46.572276 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.572278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.577394 (+ 5116us) server_negotiation.cc:407] Connection header received
0620 08:11:46.577494 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.577495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.577504 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.577532 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:46.577744 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.578487 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.582543 (+ 4056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.583090 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.583137 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.583146 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.583368 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.583407 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:11:46.583896 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.572075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:33867 (local address 127.12.132.193:35377)
0620 08:11:46.577321 (+ 5246us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.577334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:46.577422 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:46.577603 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:46.577604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:46.577606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:46.577677 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.577679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.578587 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.578588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.582436 (+ 3848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.582439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.583223 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.583224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.583293 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.583297 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:11:46.583754 (+ 457us) client_negotiation.cc:240] Negotiation successful
0620 08:11:46.583800 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5227,"thread_start_us":71,"threads_started":1}
I20260620 08:11:46.585249 26081 tablet_bootstrap.cc:492] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: No bootstrap required, opened a new log
I20260620 08:11:46.585323 26081 ts_tablet_manager.cc:1397] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260620 08:11:46.584328 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d7466f780448c4bb90f122eecdc1af" candidate_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" is_pre_election: true
W20260620 08:11:46.586228 25667 leader_election.cc:341] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d44edc0b8a845bb81c8d97d0c1b1d1f (127.12.132.194:33867): Illegal state: must be running to vote when last-logged opid is not known
I20260620 08:11:46.587430 26081 raft_consensus.cc:348] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.587574 26081 raft_consensus.cc:374] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:46.587604 26081 raft_consensus.cc:729] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d44edc0b8a845bb81c8d97d0c1b1d1f, State: Initialized, Role: FOLLOWER
I20260620 08:11:46.587709 26081 consensus_queue.cc:260] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.587984 26081 ts_tablet_manager.cc:1428] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:11:46.588097 25912 heartbeater.cc:503] Master 127.12.132.254:37511 was elected leader, sending a full tablet report...
I20260620 08:11:46.589531 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.572576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36491 (local address 127.12.132.195:33191)
0620 08:11:46.572613 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.572615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.579162 (+ 6547us) server_negotiation.cc:407] Connection header received
0620 08:11:46.579254 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.579255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.579264 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.579291 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:46.579486 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.580184 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.588538 (+ 8354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.589128 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.589177 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.589185 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.589427 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.589466 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:11:46.589984 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.572412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:33191 (local address 127.12.132.193:36491)
0620 08:11:46.579106 (+ 6694us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.579117 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:46.579187 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:46.579346 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:46.579346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:46.579347 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:46.579415 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.579416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.580278 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.580279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.581492 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:46.581495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.589271 (+ 7776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.589271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:46.589350 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.589354 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:11:46.589846 (+ 492us) client_negotiation.cc:240] Negotiation successful
0620 08:11:46.589893 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6677,"thread_start_us":215,"threads_started":1}
I20260620 08:11:46.590370 26000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d7466f780448c4bb90f122eecdc1af" candidate_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" is_pre_election: true
I20260620 08:11:46.590505 26000 raft_consensus.cc:2454] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de5513b71f7c4a06a7720ea4f19c3db3 in term 0.
I20260620 08:11:46.590767 25668 leader_election.cc:302] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ad21d6417ed4ea9834e6e0ef96b9743, de5513b71f7c4a06a7720ea4f19c3db3; no voters: 8d44edc0b8a845bb81c8d97d0c1b1d1f
I20260620 08:11:46.591017 26084 raft_consensus.cc:2781] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:11:46.591121 26084 raft_consensus.cc:482] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:11:46.591185 26084 raft_consensus.cc:3037] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:46.592069 26084 raft_consensus.cc:504] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.592373 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d7466f780448c4bb90f122eecdc1af" candidate_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f"
I20260620 08:11:46.592443 25866 raft_consensus.cc:3037] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:46.593238 25866 raft_consensus.cc:2454] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de5513b71f7c4a06a7720ea4f19c3db3 in term 1.
I20260620 08:11:46.593394 25667 leader_election.cc:302] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d44edc0b8a845bb81c8d97d0c1b1d1f, de5513b71f7c4a06a7720ea4f19c3db3; no voters:
I20260620 08:11:46.593536 26085 raft_consensus.cc:2781] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:46.593708 26085 raft_consensus.cc:686] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [term 1 LEADER]: Becoming Leader. State: Replica: de5513b71f7c4a06a7720ea4f19c3db3, State: Running, Role: LEADER
I20260620 08:11:46.593827 26085 consensus_queue.cc:237] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } }
I20260620 08:11:46.595046 26000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d7466f780448c4bb90f122eecdc1af" candidate_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743"
I20260620 08:11:46.595121 26000 raft_consensus.cc:3037] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:46.595911 26000 raft_consensus.cc:2454] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de5513b71f7c4a06a7720ea4f19c3db3 in term 1.
I20260620 08:11:46.596526 25580 catalog_manager.cc:5495] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 reported cstate change: term changed from 0 to 1, leader changed from <none> to de5513b71f7c4a06a7720ea4f19c3db3 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de5513b71f7c4a06a7720ea4f19c3db3" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 46469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 } health_report { overall_health: UNKNOWN } } }
I20260620 08:11:46.596930 26084 leader_election.cc:288] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [CANDIDATE]: Term 1 election: Requested vote from peers 8d44edc0b8a845bb81c8d97d0c1b1d1f (127.12.132.194:33867), 7ad21d6417ed4ea9834e6e0ef96b9743 (127.12.132.195:33191)
I20260620 08:11:46.605912 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.602691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34180 (local address 127.12.132.193:46469)
0620 08:11:46.602728 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:46.602729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:46.602736 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:11:46.602767 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:46.602768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:46.602784 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:46.602843 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:11:46.603008 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.603706 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.604893 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:46.605147 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:46.605211 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:46.605409 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:11:46.605529 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:11:46.605609 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:46.605821 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 08:11:46.605840 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:11:46.608816 26000 raft_consensus.cc:1261] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 1 FOLLOWER]: Refusing update from remote peer de5513b71f7c4a06a7720ea4f19c3db3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:46.609261 26091 consensus_queue.cc:1035] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ad21d6417ed4ea9834e6e0ef96b9743" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 33191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:46.609406 25866 raft_consensus.cc:1261] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 1 FOLLOWER]: Refusing update from remote peer de5513b71f7c4a06a7720ea4f19c3db3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:46.609676 26091 consensus_queue.cc:1035] T e2d7466f780448c4bb90f122eecdc1af P de5513b71f7c4a06a7720ea4f19c3db3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d44edc0b8a845bb81c8d97d0c1b1d1f" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:46.612010 26094 mvcc.cc:205] Tried to move back new op lower bound from 7298838964666535936 to 7298838964610617344. Current Snapshot: MvccSnapshot[applied={T|T < 7298838964666535936}]
I20260620 08:11:46.612915 26096 mvcc.cc:205] Tried to move back new op lower bound from 7298838964666535936 to 7298838964610617344. Current Snapshot: MvccSnapshot[applied={T|T < 7298838964666535936}]
I20260620 08:11:46.615311 25580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:46.615397 25580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:46.616223 25580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 864096ea78e14c17947649d78f82fee9: Sending DeleteTablet for 3 replicas of tablet e2d7466f780448c4bb90f122eecdc1af
I20260620 08:11:46.616662 25846 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d7466f780448c4bb90f122eecdc1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:46 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:60040
I20260620 08:11:46.616863 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d7466f780448c4bb90f122eecdc1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:46 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:34166
I20260620 08:11:46.617200 25980 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d7466f780448c4bb90f122eecdc1af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:46 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:44676
I20260620 08:11:46.617849 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25618
I20260620 08:11:46.621829 26102 tablet_replica.cc:330] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: stopping tablet replica
I20260620 08:11:46.622083 26102 raft_consensus.cc:2229] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:46.622298 26102 raft_consensus.cc:2258] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:46.622593 26104 tablet_replica.cc:330] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: stopping tablet replica
I20260620 08:11:46.622781 26104 raft_consensus.cc:2229] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:46.622867 26104 pending_rounds.cc:62] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Trying to abort 2 pending ops.
I20260620 08:11:46.622905 26104 pending_rounds.cc:68] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298838964666535936 op_type: WRITE_OP write_request { tablet_id: "e2d7466f780448c4bb90f122eecdc1af" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\356\230\331\321\006\"8\n\ntest-admin\022*\n 30444d4c565845b0bc00678f098b17ca\020\001\030\001 \001(\001" signature: "<redacted>""\256\365L\n\274\272s\276\002c\2759\017\370\2039V\334\220\226\324h\204\026\365p\326-\334i\034\365\031\266\244\236\t5\215\353\243\220Q\352\001\347\3114M@\214|\235`T\371#\213\304t\257]\232\257" signing_key_seq_num: 0 } } request_id { client_id: "23b4057de8844699adfc4943d2c6a7af" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260620 08:11:46.623267 26104 pending_rounds.cc:68] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298838964610617344 op_type: NO_OP noop_request { }
I20260620 08:11:46.623358 26104 raft_consensus.cc:2866] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260620 08:11:46.623400 26104 raft_consensus.cc:2258] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:46.623996 26104 ts_tablet_manager.cc:1905] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:46.626866 26104 ts_tablet_manager.cc:1918] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:46.627049 26104 log.cc:1196] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/e2d7466f780448c4bb90f122eecdc1af
I20260620 08:11:46.627470 26104 ts_tablet_manager.cc:1939] T e2d7466f780448c4bb90f122eecdc1af P 7ad21d6417ed4ea9834e6e0ef96b9743: Deleting consensus metadata
I20260620 08:11:46.628206 25567 catalog_manager.cc:4841] TS 7ad21d6417ed4ea9834e6e0ef96b9743 (127.12.132.195:33191): tablet e2d7466f780448c4bb90f122eecdc1af (table test-table [id=30444d4c565845b0bc00678f098b17ca]) successfully deleted
W20260620 08:11:46.628881 25934 connection.cc:535] server connection from 127.12.132.193:36491 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:11:46.628894 26102 ts_tablet_manager.cc:1905] T e2d7466f780448c4bb90f122eecdc1af P 8d44edc0b8a845bb81c8d97d0c1b1d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 08:11:46.629437 25566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:46469 (error 108)
I20260620 08:11:46.630553 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.630398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:46469 (local address 127.0.0.1:34186)
0620 08:11:46.630461 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.630502 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:46469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 08:11:46.630694 25566 catalog_manager.cc:4584] TS de5513b71f7c4a06a7720ea4f19c3db3 (127.12.132.193:46469): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2d7466f780448c4bb90f122eecdc1af: Network error: Client connection negotiation failed: client connection to 127.12.132.193:46469: connect: Connection refused (error 111)
I20260620 08:11:46.630965 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25782
I20260620 08:11:46.637181 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:46.637003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:33867 (local address 127.0.0.1:60044)
0620 08:11:46.637093 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:46.637127 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 08:11:46.637187 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25914
I20260620 08:11:46.643931 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 25547
I20260620 08:11:46.649971 12819 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 08:11:46.995134 12819 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 08:11:46.995270 12819 mini_ranger.cc:66] Stopping Ranger...
I20260620 08:11:47.340562 12819 mini_ranger.cc:68] Stopped Ranger
2026-06-20 08:11:47.340 UTC [24614] LOG: received smart shutdown request
2026-06-20 08:11:47.344 UTC [24614] LOG: background worker "logical replication launcher" (PID 24623) exited with exit code 1
2026-06-20 08:11:47.345 UTC [24618] LOG: shutting down
2026-06-20 08:11:47.559 UTC [24614] LOG: database system is shut down
2026-06-20T08:11:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50501 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:11:47 dist-test-slave-z0sp krb5kdc[26114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:11:47 dist-test-slave-z0sp krb5kdc[26114](info): set up 2 sockets
Jun 20 08:11:47 dist-test-slave-z0sp krb5kdc[26114](info): commencing operation
krb5kdc: starting...
W20260620 08:11:48.725214 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:11:48 dist-test-slave-z0sp krb5kdc[26114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:11:48.851311 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43805
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:48.934517 26127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:48.934758 26127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:48.934793 26127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:48.935876 26127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:48.935942 26127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:48.935966 26127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:48.935989 26127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:48.936015 26127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:48.937489 26127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:48.937752 26127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:48.937976 26127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:48.942850 26135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:48.942880 26133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:48.942891 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:48.942993 26127 server_base.cc:1034] running on GCE node
I20260620 08:11:48.943310 26127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:48.943590 26127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:48.944748 26127 hybrid_clock.cc:648] HybridClock initialized: now 1781943108944723 us; error 53 us; skew 500 ppm
Jun 20 08:11:48 dist-test-slave-z0sp krb5kdc[26114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:48.951566 26127 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:48.952222 26127 webserver.cc:460] Webserver started at http://127.12.132.254:43599/ using document root <none> and password file <none>
I20260620 08:11:48.952572 26127 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:48.952682 26127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:48.952850 26127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:48.954121 26127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "2a42050fd91d4ad1ade7858d34d0f1d2"
format_stamp: "Formatted at 2026-06-20 08:11:48 on dist-test-slave-z0sp"
server_key: "cbbc8818fff6e3b6ad9bc0c3e457c69f"
server_key_iv: "75ce69fd66d6a858502871dfa89dc40e"
server_key_version: "encryptionkey@0"
I20260620 08:11:48.954528 26127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "2a42050fd91d4ad1ade7858d34d0f1d2"
format_stamp: "Formatted at 2026-06-20 08:11:48 on dist-test-slave-z0sp"
server_key: "cbbc8818fff6e3b6ad9bc0c3e457c69f"
server_key_iv: "75ce69fd66d6a858502871dfa89dc40e"
server_key_version: "encryptionkey@0"
I20260620 08:11:48.956195 26127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:11:48.957332 26142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:48.957615 26127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:48.957739 26127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "2a42050fd91d4ad1ade7858d34d0f1d2"
format_stamp: "Formatted at 2026-06-20 08:11:48 on dist-test-slave-z0sp"
server_key: "cbbc8818fff6e3b6ad9bc0c3e457c69f"
server_key_iv: "75ce69fd66d6a858502871dfa89dc40e"
server_key_version: "encryptionkey@0"
I20260620 08:11:48.957842 26127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:48.969031 26127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:48.971287 26127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:48.971460 26127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:48.975631 26127 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:43805
I20260620 08:11:48.975674 26194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:43805 every 8 connection(s)
I20260620 08:11:48.976015 26127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:48.976431 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26127
I20260620 08:11:48.976560 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:11:48.976747 12819 external_mini_cluster.cc:1428] Setting key e196a232d5dcc99c87b1eae9ce7decb5
I20260620 08:11:48.977082 26195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:48.979127 26195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.132.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.253@KRBTEST.COM" created.
Jun 20 08:11:48 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:48.991693 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:48.979889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.12.132.254:43805)
0620 08:11:48.980087 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:48.980095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:48.980166 (+ 71us) server_negotiation.cc:407] Connection header received
0620 08:11:48.980999 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:48.981001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:48.981018 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:48.981141 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:48.981740 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:48.982519 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:48.983821 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:48.984128 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:48.984260 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:48.988374 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:48.988382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:48.988382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:48.988390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:48.990099 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:48.990501 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:48.990504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:48.990505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:48.990583 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:48.990810 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:48.990811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:48.990811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:48.991023 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:48.991112 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:48.991286 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 08:11:48.991450 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
I20260620 08:11:48.991701 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:48.979862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43805 (local address 127.0.0.1:58726)
0620 08:11:48.980041 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:48.980087 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:48.980843 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:48.981162 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:48.981165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:48.981451 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:48.981612 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:48.981619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:48.982722 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:48.982725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:48.983697 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:48.983703 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:48.984213 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:48.984214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:48.984283 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:48.984913 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:48.984922 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:48.988279 (+ 3357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:48.990207 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:48.990213 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:48.990215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:48.990407 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:48.990650 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:48.990650 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:48.990651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:48.990741 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:48.991093 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:48.991098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:48.991164 (+ 66us) client_negotiation.cc:769] Sending connection context
0620 08:11:48.991260 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 08:11:48.991498 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":80,"thread_start_us":83,"threads_started":1}
I20260620 08:11:48.992571 26195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:11:48.992921 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:48.992815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56318)
0620 08:11:48.992863 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:48.992904 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 08:11:48.993044 26144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
I20260620 08:11:48.993463 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:48.993375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56320)
0620 08:11:48.993419 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:48.993442 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 08:11:48.993561 26195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:42103: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
I20260620 08:11:49.004050 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:42103
--webserver_interface=127.12.132.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:49.042812 26195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } attempt: 1
I20260620 08:11:49.043392 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.043222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56326)
0620 08:11:49.043306 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.043361 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 08:11:49.043838 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.043692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56332)
0620 08:11:49.043784 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.043814 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260620 08:11:49.043953 26195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:42103: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
W20260620 08:11:49.086243 26202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:49.086481 26202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:49.086520 26202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:49.087641 26202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:49.087702 26202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:49.087726 26202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:49.087746 26202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:49.087785 26202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:49.089351 26202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.132.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:49.089612 26202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:49.089821 26202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:49.094686 26210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:49.094686 26207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:49.094707 26208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:49.095132 26202 server_base.cc:1034] running on GCE node
I20260620 08:11:49.095417 26202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:49.095741 26202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:49.096925 26202 hybrid_clock.cc:648] HybridClock initialized: now 1781943109096900 us; error 46 us; skew 500 ppm
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:49.099395 26202 init.cc:377] Logged in from keytab as kudu/127.12.132.253@KRBTEST.COM (short username kudu)
I20260620 08:11:49.099864 26202 webserver.cc:460] Webserver started at http://127.12.132.253:33899/ using document root <none> and password file <none>
I20260620 08:11:49.100059 26202 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:49.100101 26202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:49.100188 26202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:49.101223 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data/instance:
uuid: "bd1c4df43a45439fa2ff812a35f6677a"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "5cd13c059fcbbe62e7990f7005645cec"
server_key_iv: "83468ab97cb0a20a5ec917d3f90b7d14"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.101523 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal/instance:
uuid: "bd1c4df43a45439fa2ff812a35f6677a"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "5cd13c059fcbbe62e7990f7005645cec"
server_key_iv: "83468ab97cb0a20a5ec917d3f90b7d14"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.103086 26202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:49.103889 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:49.104197 26202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:49.104287 26202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "bd1c4df43a45439fa2ff812a35f6677a"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "5cd13c059fcbbe62e7990f7005645cec"
server_key_iv: "83468ab97cb0a20a5ec917d3f90b7d14"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.104387 26202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:49.112113 26195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } attempt: 2
I20260620 08:11:49.112483 26202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:49.112591 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.112414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56344)
0620 08:11:49.112472 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.112557 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:11:49.112952 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.112841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56358)
0620 08:11:49.112882 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.112919 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260620 08:11:49.113054 26195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:42103: Network error: Client connection negotiation failed: client connection to 127.12.132.253:42103: connect: Connection refused (error 111)
I20260620 08:11:49.114967 26202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:49.115118 26202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:49.119026 26202 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.253:42103
I20260620 08:11:49.119061 26269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.253:42103 every 8 connection(s)
I20260620 08:11:49.119400 26202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
I20260620 08:11:49.120231 26270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:49.121790 26270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:49.128916 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26202
I20260620 08:11:49.129032 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-1/wal/instance
I20260620 08:11:49.129236 12819 external_mini_cluster.cc:1428] Setting key 76fb162fb5e19448cdb3255a2f4e76c6
I20260620 08:11:49.131485 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.122490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.12.132.254:43805)
0620 08:11:49.122567 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.122569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.122785 (+ 216us) server_negotiation.cc:407] Connection header received
0620 08:11:49.123225 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.123227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.123238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.123304 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.123876 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.124355 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.125487 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.125757 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.125861 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.128184 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.128186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.128186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.128194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.130073 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.130503 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.130504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.130505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.130564 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.130819 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.130819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.130820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.130939 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.131012 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.131282 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.131371 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:11:49.131693 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.122446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43805 (local address 127.0.0.1:58742)
0620 08:11:49.122608 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.122646 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.123112 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.123357 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.123360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.123609 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.123788 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.123792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.124474 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.124475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.125334 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.125337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.125886 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.125887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.126064 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.126618 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.126626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.128100 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.130193 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.130200 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.130202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.130426 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.130630 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.130631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.130631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.130747 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.131004 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.131010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.131145 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.131251 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.131405 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260620 08:11:49.132266 26270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.132.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.252@KRBTEST.COM" created.
I20260620 08:11:49.141273 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.132584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56372)
0620 08:11:49.132817 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.132829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.132899 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.133048 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.133049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.133216 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.133278 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.133281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.134108 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.134109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.135005 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.135008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.135477 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.135479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.135558 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.136068 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.136074 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.137569 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.140007 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.140008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.140008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.140238 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.140465 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.140466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.140467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.140556 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.140848 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.140849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.140897 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.140971 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.141087 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260620 08:11:49.141294 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.132653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.12.132.253:42103)
0620 08:11:49.132822 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.132825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.132906 (+ 81us) server_negotiation.cc:407] Connection header received
0620 08:11:49.132947 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.132950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.132966 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.133039 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.133343 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.134018 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.135108 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.135397 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.135458 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.137741 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.137762 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.137762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.137770 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.139909 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.140340 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.140343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.140343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.140400 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.140633 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.140634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.140634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.140784 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.140849 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.141006 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.141191 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260620 08:11:49.142035 26270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:11:49.142390 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.142276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47922)
0620 08:11:49.142318 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.142365 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 08:11:49.142544 26221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
I20260620 08:11:49.142961 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.142877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47928)
0620 08:11:49.142920 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.142945 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 08:11:49.143066 26270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:36227: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
I20260620 08:11:49.156406 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:36227
--webserver_interface=127.12.132.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:49.192414 26270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } attempt: 1
I20260620 08:11:49.193125 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.192966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47940)
0620 08:11:49.193043 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.193091 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 08:11:49.193493 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.193398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47946)
0620 08:11:49.193439 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.193464 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 08:11:49.193583 26270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:36227: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
I20260620 08:11:49.204226 26195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } attempt: 3
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:11:49.212059 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.204667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56388)
0620 08:11:49.204729 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.204742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.204814 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.204967 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.204968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.205135 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.205202 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.205203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.205885 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.205886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.206381 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.206383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.206784 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.206785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.206874 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.207390 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.207395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.208691 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.210751 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.210752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.210753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.210924 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.211200 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.211203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.211203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.211319 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.211688 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.211690 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.211756 (+ 66us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.211837 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.211968 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:11:49.212077 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.204765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.12.132.253:42103)
0620 08:11:49.204839 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.204841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.204852 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:11:49.204885 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.204885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.204897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.204963 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.205289 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.205800 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.206474 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.206712 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.206788 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.208777 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.208779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.208779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.208788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.210648 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.210996 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.210997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.210998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.211065 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.211424 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.211425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.211425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.211562 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.211638 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.211878 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.211979 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 08:11:49.212620 26195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:11:49.213065 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.212929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47948)
0620 08:11:49.212995 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.213034 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 08:11:49.213455 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.213350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47964)
0620 08:11:49.213390 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.213428 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260620 08:11:49.213553 26195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:36227: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
W20260620 08:11:49.240485 26277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:49.240736 26277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:49.240777 26277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:49.241936 26277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:49.241997 26277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:49.242023 26277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:49.242056 26277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:49.242084 26277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:49.243523 26277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43805,127.12.132.253:42103,127.12.132.252:36227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:36227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.132.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:49.243778 26277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:49.243993 26277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:11:49.244694 26195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } attempt: 1
I20260620 08:11:49.245198 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.245034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47976)
0620 08:11:49.245110 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.245155 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 08:11:49.245561 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.245475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:47986)
0620 08:11:49.245518 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.245544 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 08:11:49.245656 26195 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:36227: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
W20260620 08:11:49.249063 26283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:49.249075 26282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:49.249176 26285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:49.249310 26277 server_base.cc:1034] running on GCE node
I20260620 08:11:49.249830 26277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:49.250310 26277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:49.251494 26277 hybrid_clock.cc:648] HybridClock initialized: now 1781943109251478 us; error 45 us; skew 500 ppm
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:49.253988 26277 init.cc:377] Logged in from keytab as kudu/127.12.132.252@KRBTEST.COM (short username kudu)
I20260620 08:11:49.254449 26277 webserver.cc:460] Webserver started at http://127.12.132.252:33867/ using document root <none> and password file <none>
I20260620 08:11:49.254631 26277 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:49.254670 26277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:49.254755 26277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:49.255749 26277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data/instance:
uuid: "390877977e2d437289eca0d3175545b8"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "64033e880991080090835773445560ab"
server_key_iv: "6f5b4d0c8b9ad3afee442e22a53fe321"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.256043 26277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal/instance:
uuid: "390877977e2d437289eca0d3175545b8"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "64033e880991080090835773445560ab"
server_key_iv: "6f5b4d0c8b9ad3afee442e22a53fe321"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.257480 26277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:11:49.258570 26292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:49.258919 26277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:49.259047 26277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "390877977e2d437289eca0d3175545b8"
format_stamp: "Formatted at 2026-06-20 08:11:49 on dist-test-slave-z0sp"
server_key: "64033e880991080090835773445560ab"
server_key_iv: "6f5b4d0c8b9ad3afee442e22a53fe321"
server_key_version: "encryptionkey@0"
I20260620 08:11:49.259127 26277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:49.261761 26270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } attempt: 2
I20260620 08:11:49.262164 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.262046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:48000)
0620 08:11:49.262104 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.262139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:11:49.262487 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.262398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:48010)
0620 08:11:49.262437 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.262460 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260620 08:11:49.262567 26270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:36227: Network error: Client connection negotiation failed: client connection to 127.12.132.252:36227: connect: Connection refused (error 111)
I20260620 08:11:49.278383 26277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:49.280825 26277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:49.280987 26277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:49.286742 26277 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.252:36227
I20260620 08:11:49.286834 26344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.252:36227 every 8 connection(s)
I20260620 08:11:49.287391 26277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
I20260620 08:11:49.288399 26345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:49.290095 26345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:11:49.291580 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26277
I20260620 08:11:49.291726 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781942949664683-12819-0/minicluster-data/master-2/wal/instance
I20260620 08:11:49.291920 12819 external_mini_cluster.cc:1428] Setting key 4e2914a223bb222abaa97d596e7f4a81
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:11:49.300765 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.290857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.12.132.254:43805)
0620 08:11:49.290933 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.290936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.291173 (+ 237us) server_negotiation.cc:407] Connection header received
0620 08:11:49.291634 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.291636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.291648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.291735 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.292616 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.293105 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.294047 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.294293 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.294399 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.296884 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.296885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.296886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.296897 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.298959 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.299665 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.299666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.299666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.299742 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.299999 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.300000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.300000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.300160 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.300220 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.300428 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.300561 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 08:11:49.300839 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.292327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.12.132.252:36227)
0620 08:11:49.292570 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.292574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.292595 (+ 21us) server_negotiation.cc:407] Connection header received
0620 08:11:49.292641 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.292644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.292662 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.292729 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.293314 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.294032 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.294750 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.295056 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.295132 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.297560 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.297568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.297568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.297575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.299320 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.299704 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.299707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.299707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.299799 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.300000 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.300001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.300001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.300227 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.300402 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.300459 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.300692 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":212,"thread_start_us":104,"threads_started":1}
I20260620 08:11:49.300845 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.290854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43805 (local address 127.0.0.1:58746)
0620 08:11:49.291040 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.291080 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.291540 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.291824 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.291828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.292191 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.292522 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.292528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.293197 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.293198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.293963 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.293965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.294408 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.294409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.294556 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.295098 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.295106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.296780 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.299094 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.299109 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.299112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.299479 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.299819 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.299821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.299821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.299930 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.300226 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.300232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.300303 (+ 71us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.300399 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.300593 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260620 08:11:49.301779 26345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:49.309430 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.302106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.12.132.253:42103)
0620 08:11:49.302151 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.302153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.302160 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:11:49.302185 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.302186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.302195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.302241 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.302611 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.303200 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.304002 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.304285 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.304343 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.306046 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.306047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.306047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.306054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.308072 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308543 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.308544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.308545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.308625 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308835 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.308836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.308836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.308980 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.309055 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.309228 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.309318 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:11:49.309429 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.302034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:42103 (local address 127.0.0.1:56398)
0620 08:11:49.302079 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.302091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.302131 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.302284 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.302284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.302442 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.302505 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.302507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.303289 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.303290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.303908 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.303910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.304357 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.304358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.304444 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.304811 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.304815 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.305959 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.308185 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308187 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.308187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.308406 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.308701 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.308703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.308771 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.309046 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.309047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.309102 (+ 55us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.309199 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.309320 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 08:11:49.309731 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.301550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.12.132.254:43805)
0620 08:11:49.301639 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.301641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.301647 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:11:49.301665 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.301665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.301674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.301724 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.302235 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.302828 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.303611 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.303898 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.303983 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.306523 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.306524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.306524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.306531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.308050 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308480 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.308481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.308481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.308572 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.308789 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.308789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.308790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.308903 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.309529 (+ 626us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.309576 (+ 47us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.309662 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 08:11:49.310313 26345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:11:49.318413 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.310655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48028 (local address 127.12.132.252:36227)
0620 08:11:49.310700 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.310702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.310748 (+ 46us) server_negotiation.cc:407] Connection header received
0620 08:11:49.310821 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.310822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.310831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.310885 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.311142 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.311652 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.312310 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.312590 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.312686 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.315109 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.315111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.315111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.315120 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.316902 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.317334 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.317336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.317336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.317400 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.317693 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.317694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.317694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.317821 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.317899 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.318150 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.318269 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:11:49.318656 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.310707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.12.132.253:42103)
0620 08:11:49.310764 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.310767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.310806 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:11:49.310872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.310873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.310882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.310945 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.311401 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.311856 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.312521 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.312723 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.312802 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.315689 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.315690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.315691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.315699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.317389 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.317825 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.317826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.317827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.317894 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.318100 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.318101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.318101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.318218 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.318417 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.318466 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.318538 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:11:49.318478 26347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.310604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:48028)
0620 08:11:49.310685 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.310699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.310766 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.310888 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.310889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.311035 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.311070 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.311071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.311743 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.311744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.312220 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.312221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.312696 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.312697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.312749 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.313195 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.313200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.314960 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.316983 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.316985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.316986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.317224 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.317551 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.317552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.317552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.317628 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.317898 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.317899 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.317941 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.318113 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.318330 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":99}
I20260620 08:11:49.320559 26345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: Bootstrap starting.
I20260620 08:11:49.320752 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:11:49.320768 26195 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } attempt: 2
I20260620 08:11:49.321650 26345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:49.321996 26345 log.cc:824] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:49.323035 26345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: No bootstrap required, opened a new log
I20260620 08:11:49.324883 26345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.325026 26345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:49.325088 26345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390877977e2d437289eca0d3175545b8, State: Initialized, Role: FOLLOWER
I20260620 08:11:49.325256 26345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.325516 26359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.325578 26345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:49.326120 26359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:11:49.327993 26370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:11:49.328111 26370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:11:49.328430 26277 master_runner.cc:186] Error getting master registration for 127.12.132.254:43805: OK, instance_id { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" instance_seqno: 1781943108974478 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:11:49.330796 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.322337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.12.132.253:42103)
0620 08:11:49.322375 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.322376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.322771 (+ 395us) server_negotiation.cc:407] Connection header received
0620 08:11:49.322829 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.322830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.322840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.322896 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.323475 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.324134 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.325076 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.325372 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.325410 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.326935 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.326936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.326937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.326943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.329074 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.329461 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.329462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.329462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.329522 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.329993 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.329994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.329994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.330117 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.330204 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.330591 (+ 387us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.330668 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:11:49.330821 26277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943108, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:11:49.332986 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.322468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.12.132.254:43805)
0620 08:11:49.322513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.322514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.322551 (+ 37us) server_negotiation.cc:407] Connection header received
0620 08:11:49.322651 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.322652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.322662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.322694 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.323413 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.323903 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.324993 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.325286 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.325326 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.326929 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.326930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.326931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.326937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.331555 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.331957 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.331959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.331959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.332022 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.332247 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.332248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.332248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.332383 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.332483 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.332799 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.332876 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:11:49.334525 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.321048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:48040)
0620 08:11:49.321094 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.321104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.321141 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.322401 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.322402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.322567 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.322615 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.322618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.327980 (+ 5362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.327981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.328872 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.328876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.329373 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.329374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.329436 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.329830 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.329835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.331290 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.333423 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.333423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.333424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.333626 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.333840 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.333841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.333841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.333905 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.334220 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.334221 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.334265 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.334346 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.334437 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 08:11:49.334525 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.321091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48040 (local address 127.12.132.252:36227)
0620 08:11:49.322253 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.322255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.322265 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:11:49.322286 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.322287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.322298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.325645 (+ 3347us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.325679 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.327880 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.328981 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.329280 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.329580 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.331382 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.331383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.331384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.331390 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.333366 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.333707 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.333708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.333708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.333765 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.333969 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.333970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.333970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.334135 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.334215 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.334370 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.334440 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142}
I20260620 08:11:49.336382 26195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: Bootstrap starting.
I20260620 08:11:49.337121 26195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:49.337435 26195 log.cc:824] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:49.338630 26195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: No bootstrap required, opened a new log
I20260620 08:11:49.339529 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.322631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.12.132.252:36227)
0620 08:11:49.329655 (+ 7024us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.329656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.329662 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:11:49.329704 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.329705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.329716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.332564 (+ 2848us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.332613 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.334009 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.334934 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.335202 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.335276 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.336428 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.336429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.336430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.336436 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.338396 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.338747 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.338748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.338748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.338806 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.339004 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.339005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.339006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.339113 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.339204 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.339327 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.339402 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7004,"thread_start_us":90,"threads_started":1}
I20260620 08:11:49.340952 26195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.341094 26195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:49.341127 26195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a42050fd91d4ad1ade7858d34d0f1d2, State: Initialized, Role: FOLLOWER
I20260620 08:11:49.341234 26195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.341527 26195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:49.342998 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.343236 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:11:49.343459 26384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:11:49.343521 26384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:11:49.344062 26127 master_runner.cc:186] Error getting master registration for 127.12.132.253:42103: OK, instance_id { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" instance_seqno: 1781943109118016 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:11:49.346487 26127 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:11:49.353729 26270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } attempt: 3
Jun 20 08:11:49 dist-test-slave-z0sp krb5kdc[26114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943109, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:11:49.362690 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.354160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:36227 (local address 127.0.0.1:48056)
0620 08:11:49.354227 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:49.354240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:49.354286 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:49.354464 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:49.354465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:49.354700 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:49.354773 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.354775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.355876 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.355877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.356748 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:49.356750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.357260 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.357261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:49.357319 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.357825 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:49.357831 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:49.359451 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:49.361432 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.361433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.361433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.361675 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.361986 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:49.361987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:49.361988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:49.362052 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:49.362365 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:49.362366 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:49.362414 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 08:11:49.362487 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 08:11:49.362583 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 08:11:49.362705 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:49.354219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.12.132.252:36227)
0620 08:11:49.354262 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:49.354263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:49.354276 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:11:49.354368 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:49.354369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:49.354379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:49.354442 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:49.354864 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.355723 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.356858 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:49.357181 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:49.357254 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:49.359542 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:49.359543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:49.359543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:49.359551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:49.361315 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.361818 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.361819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.361819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.361883 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:49.362118 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:49.362119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:49.362119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:49.362261 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:49.362366 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:49.362523 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 08:11:49.362609 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:11:49.364833 26270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: Bootstrap starting.
I20260620 08:11:49.365597 26270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:49.365922 26270 log.cc:824] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:49.366793 26270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: No bootstrap required, opened a new log
I20260620 08:11:49.368861 26270 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.368988 26270 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:49.369021 26270 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd1c4df43a45439fa2ff812a35f6677a, State: Initialized, Role: FOLLOWER
I20260620 08:11:49.369241 26270 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.369611 26270 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:49.369576 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.369798 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:11:49.371460 26398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:11:49.371558 26398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:11:49.373485 26202 master_runner.cc:263] No leader master found from master bd1c4df43a45439fa2ff812a35f6677a
I20260620 08:11:49.373557 26202 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:11:49.398806 26373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:49.398938 26373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.399180 26373 leader_election.cc:288] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd1c4df43a45439fa2ff812a35f6677a (127.12.132.253:42103), 390877977e2d437289eca0d3175545b8 (127.12.132.252:36227)
I20260620 08:11:49.399693 26244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd1c4df43a45439fa2ff812a35f6677a" is_pre_election: true
I20260620 08:11:49.399845 26244 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a42050fd91d4ad1ade7858d34d0f1d2 in term 0.
I20260620 08:11:49.400033 26144 leader_election.cc:302] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a42050fd91d4ad1ade7858d34d0f1d2, bd1c4df43a45439fa2ff812a35f6677a; no voters:
I20260620 08:11:49.400280 26373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:11:49.400348 26373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:11:49.400373 26373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:49.401340 26373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.401461 26373 leader_election.cc:288] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [CANDIDATE]: Term 1 election: Requested vote from peers bd1c4df43a45439fa2ff812a35f6677a (127.12.132.253:42103), 390877977e2d437289eca0d3175545b8 (127.12.132.252:36227)
I20260620 08:11:49.401744 26244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd1c4df43a45439fa2ff812a35f6677a"
I20260620 08:11:49.401819 26244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:49.402737 26244 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a42050fd91d4ad1ade7858d34d0f1d2 in term 1.
I20260620 08:11:49.402923 26144 leader_election.cc:302] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a42050fd91d4ad1ade7858d34d0f1d2, bd1c4df43a45439fa2ff812a35f6677a; no voters:
I20260620 08:11:49.403311 26319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390877977e2d437289eca0d3175545b8" is_pre_election: true
I20260620 08:11:49.403481 26319 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a42050fd91d4ad1ade7858d34d0f1d2 in term 0.
I20260620 08:11:49.403586 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390877977e2d437289eca0d3175545b8"
I20260620 08:11:49.403635 26318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:49.404450 26318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a42050fd91d4ad1ade7858d34d0f1d2 in term 1.
I20260620 08:11:49.406076 26373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:49.406262 26373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [term 1 LEADER]: Becoming Leader. State: Replica: 2a42050fd91d4ad1ade7858d34d0f1d2, State: Running, Role: LEADER
I20260620 08:11:49.406363 26373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } }
I20260620 08:11:49.406767 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a42050fd91d4ad1ade7858d34d0f1d2. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.406832 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:49.407222 26402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:49.407451 26402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:49.409634 26318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [term 1 FOLLOWER]: Refusing update from remote peer 2a42050fd91d4ad1ade7858d34d0f1d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:49.409994 26373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:49.411268 26359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a42050fd91d4ad1ade7858d34d0f1d2. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.411348 26359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:11:49.411785 26244 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [term 1 FOLLOWER]: Refusing update from remote peer 2a42050fd91d4ad1ade7858d34d0f1d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:49.412035 26373 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:49.413213 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a42050fd91d4ad1ade7858d34d0f1d2. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.413304 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:11:49.414332 26400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.414441 26400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:49.414881 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.414979 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:11:49.415238 26403 mvcc.cc:205] Tried to move back new op lower bound from 7298838976139464704 to 7298838976129003520. Current Snapshot: MvccSnapshot[applied={T|T < 7298838976139464704}]
I20260620 08:11:49.416448 26406 mvcc.cc:205] Tried to move back new op lower bound from 7298838976139464704 to 7298838976129003520. Current Snapshot: MvccSnapshot[applied={T|T < 7298838976139464704}]
I20260620 08:11:49.416795 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.416865 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:49.417358 26359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a42050fd91d4ad1ade7858d34d0f1d2" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43805 } } peers { permanent_uuid: "bd1c4df43a45439fa2ff812a35f6677a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 42103 } } peers { permanent_uuid: "390877977e2d437289eca0d3175545b8" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 36227 } } }
I20260620 08:11:49.417454 26359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:11:49.418960 26402 catalog_manager.cc:1347] Generated new cluster ID: 67c3385c87ad448698e0098ce8cd2492
I20260620 08:11:49.419003 26402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:11:49.424917 26404 mvcc.cc:205] Tried to move back new op lower bound from 7298838976139464704 to 7298838976129003520. Current Snapshot: MvccSnapshot[applied={T|T < 7298838976139464704}]
I20260620 08:11:49.440655 26402 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:11:49.441495 26402 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:49.452176 26402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a42050fd91d4ad1ade7858d34d0f1d2: Generated new TSK 0
I20260620 08:11:49.452454 26402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:11:50.328681 26370 catalog_manager.cc:1259] Loaded cluster ID: 67c3385c87ad448698e0098ce8cd2492
I20260620 08:11:50.328765 26370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: loading cluster ID for follower catalog manager: success
I20260620 08:11:50.329648 26370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: acquiring CA information for follower catalog manager: success
I20260620 08:11:50.329952 26370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 390877977e2d437289eca0d3175545b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:11:50.372037 26398 catalog_manager.cc:1259] Loaded cluster ID: 67c3385c87ad448698e0098ce8cd2492
I20260620 08:11:50.372116 26398 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: loading cluster ID for follower catalog manager: success
I20260620 08:11:50.372990 26398 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: acquiring CA information for follower catalog manager: success
I20260620 08:11:50.373293 26398 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd1c4df43a45439fa2ff812a35f6677a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:11:50.511992 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26127
I20260620 08:11:50.517622 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26202
I20260620 08:11:50.523420 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26277
2026-06-20T08:11:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2855 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:11:50 dist-test-slave-z0sp krb5kdc[26419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:11:50 dist-test-slave-z0sp krb5kdc[26419](info): set up 2 sockets
Jun 20 08:11:50 dist-test-slave-z0sp krb5kdc[26419](info): commencing operation
krb5kdc: starting...
W20260620 08:11:51.569031 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:11:51.692129 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:35737
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:35737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:51.774207 26432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:51.774416 26432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:51.774438 26432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:51.775550 26432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:51.775593 26432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:51.775609 26432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:51.775621 26432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:51.775633 26432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:51.777125 26432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:35737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:51.777308 26432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:51.777482 26432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:51.782178 26438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:51.782238 26437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:51.782589 26432 server_base.cc:1034] running on GCE node
W20260620 08:11:51.782358 26440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:51.782925 26432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:51.783243 26432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:51.784432 26432 hybrid_clock.cc:648] HybridClock initialized: now 1781943111784414 us; error 40 us; skew 500 ppm
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943111, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:51.786921 26432 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:51.787457 26432 webserver.cc:460] Webserver started at http://127.12.132.254:38931/ using document root <none> and password file <none>
I20260620 08:11:51.787701 26432 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:51.787765 26432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:51.787926 26432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:51.789067 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ad992895bb4845368b7790d9accbd548"
format_stamp: "Formatted at 2026-06-20 08:11:51 on dist-test-slave-z0sp"
server_key: "829c768646e86004c828f725b5848354"
server_key_iv: "cf6ba2e6d7204651b3363f92af10d0fb"
server_key_version: "encryptionkey@0"
I20260620 08:11:51.789441 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ad992895bb4845368b7790d9accbd548"
format_stamp: "Formatted at 2026-06-20 08:11:51 on dist-test-slave-z0sp"
server_key: "829c768646e86004c828f725b5848354"
server_key_iv: "cf6ba2e6d7204651b3363f92af10d0fb"
server_key_version: "encryptionkey@0"
I20260620 08:11:51.790771 26432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:51.791512 26447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:51.791673 26432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:11:51.791764 26432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ad992895bb4845368b7790d9accbd548"
format_stamp: "Formatted at 2026-06-20 08:11:51 on dist-test-slave-z0sp"
server_key: "829c768646e86004c828f725b5848354"
server_key_iv: "cf6ba2e6d7204651b3363f92af10d0fb"
server_key_version: "encryptionkey@0"
I20260620 08:11:51.791838 26432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:51.810133 26432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:51.812418 26432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:51.812606 26432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:51.816931 26432 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:35737
I20260620 08:11:51.817046 26499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:35737 every 8 connection(s)
I20260620 08:11:51.817335 26432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:51.818161 26500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:51.820349 26500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548: Bootstrap starting.
I20260620 08:11:51.821014 26500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:51.821309 26500 log.cc:824] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:51.822042 26500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548: No bootstrap required, opened a new log
I20260620 08:11:51.823626 26500 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } }
I20260620 08:11:51.823769 26500 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:51.823794 26500 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad992895bb4845368b7790d9accbd548, State: Initialized, Role: FOLLOWER
I20260620 08:11:51.823961 26500 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } }
I20260620 08:11:51.824023 26500 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:51.824071 26500 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:51.824121 26500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:51.824690 26500 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } }
I20260620 08:11:51.824793 26500 leader_election.cc:302] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad992895bb4845368b7790d9accbd548; no voters:
I20260620 08:11:51.824980 26500 leader_election.cc:288] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:11:51.825034 26503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:51.825222 26500 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:51.825241 26503 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [term 1 LEADER]: Becoming Leader. State: Replica: ad992895bb4845368b7790d9accbd548, State: Running, Role: LEADER
I20260620 08:11:51.825321 26503 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } }
I20260620 08:11:51.825721 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad992895bb4845368b7790d9accbd548. Latest consensus state: current_term: 1 leader_uuid: "ad992895bb4845368b7790d9accbd548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } } }
I20260620 08:11:51.825796 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:51.825858 26504 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad992895bb4845368b7790d9accbd548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad992895bb4845368b7790d9accbd548" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 35737 } } }
I20260620 08:11:51.825941 26504 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:51.826259 26512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:51.826820 26512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:51.827103 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26432
I20260620 08:11:51.827193 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:11:51.827379 12819 external_mini_cluster.cc:1428] Setting key a8b65cac6cc24a2ee202dd0f9faea97e
I20260620 08:11:51.828557 26512 catalog_manager.cc:1347] Generated new cluster ID: cbca21f0686f4a598d0df6b2e638ff02
I20260620 08:11:51.828605 26512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:51.838878 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:51.827849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.12.132.254:35737)
0620 08:11:51.828180 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:51.828189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:51.828217 (+ 28us) server_negotiation.cc:407] Connection header received
0620 08:11:51.828330 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:51.828337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:51.828389 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:51.828581 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:51.829316 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:51.830751 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:51.831792 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:51.832404 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:51.832542 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:51.835192 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:51.835196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:51.835197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:51.835209 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:51.837288 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:51.837647 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:51.837650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:51.837650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:51.837752 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:51.837955 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:51.837956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:51.837956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:51.838270 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:51.838424 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:51.838502 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:11:51.838639 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":79,"threads_started":1}
I20260620 08:11:51.839732 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:11:51.842099 26512 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:11:51.842572 26512 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:51.851243 26512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad992895bb4845368b7790d9accbd548: Generated new TSK 0
I20260620 08:11:51.851424 26512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:11:51.894429 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26432
2026-06-20T08:11:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1370 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26527](info): set up 2 sockets
Jun 20 08:11:51 dist-test-slave-z0sp krb5kdc[26527](info): commencing operation
krb5kdc: starting...
W20260620 08:11:52.937058 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:11:53.057320 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:53.139223 26540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:53.139452 26540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:53.139489 26540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:53.140626 26540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:53.140688 26540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:53.140712 26540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:53.140749 26540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:53.140772 26540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:53.142206 26540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:53.142462 26540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:53.142669 26540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:53.147315 26545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:53.147346 26540 server_base.cc:1034] running on GCE node
W20260620 08:11:53.147320 26546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.147315 26548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:53.147790 26540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:53.148284 26540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:53.149586 26540 hybrid_clock.cc:648] HybridClock initialized: now 1781943113149530 us; error 65 us; skew 500 ppm
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:53.151988 26540 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:53.152411 26540 webserver.cc:460] Webserver started at http://127.12.132.254:33123/ using document root <none> and password file <none>
I20260620 08:11:53.152659 26540 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:53.152707 26540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:53.152832 26540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:53.153810 26540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "091671cfb3cb4096b1ba00aba8df4cd8"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "3d8d52633a509b253fe8ff08554ebff3"
server_key_iv: "60ad6ec65bcd4e8d721f4c43e6039294"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.154146 26540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "091671cfb3cb4096b1ba00aba8df4cd8"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "3d8d52633a509b253fe8ff08554ebff3"
server_key_iv: "60ad6ec65bcd4e8d721f4c43e6039294"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.155398 26540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:11:53.156117 26555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.156316 26540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:53.156400 26540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "091671cfb3cb4096b1ba00aba8df4cd8"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "3d8d52633a509b253fe8ff08554ebff3"
server_key_iv: "60ad6ec65bcd4e8d721f4c43e6039294"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.156476 26540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:53.169196 26540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:53.172086 26540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:53.172277 26540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:53.176271 26540 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:43939
I20260620 08:11:53.176389 26607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:43939 every 8 connection(s)
I20260620 08:11:53.176698 26540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:53.177659 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:53.179793 26608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap starting.
I20260620 08:11:53.180321 26608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:53.180585 26608 log.cc:824] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:53.181242 26608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: No bootstrap required, opened a new log
I20260620 08:11:53.182747 26608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:53.182972 26608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:53.182996 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26540
I20260620 08:11:53.183014 26608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Initialized, Role: FOLLOWER
I20260620 08:11:53.183094 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:11:53.183198 26608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:53.183285 26608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:53.183288 12819 external_mini_cluster.cc:1428] Setting key 17a77849107ab10f15c2d5227f6495d9
I20260620 08:11:53.183321 26608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:53.183377 26608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:53.184413 26608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:53.184561 26608 leader_election.cc:302] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091671cfb3cb4096b1ba00aba8df4cd8; no voters:
I20260620 08:11:53.184825 26608 leader_election.cc:288] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:11:53.184882 26613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:53.185122 26608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:53.185160 26613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 LEADER]: Becoming Leader. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Running, Role: LEADER
I20260620 08:11:53.185386 26613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:53.185752 26613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091671cfb3cb4096b1ba00aba8df4cd8. Latest consensus state: current_term: 1 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:53.185756 26615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:53.185909 26615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:53.186126 26613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:53.186152 26622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:53.186719 26622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:53.188405 26622 catalog_manager.cc:1347] Generated new cluster ID: fa1a47fa26574ed78b3db380fb4944f1
I20260620 08:11:53.188457 26622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:53.195322 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.183996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.12.132.254:43939)
0620 08:11:53.184243 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.184254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.184293 (+ 39us) server_negotiation.cc:407] Connection header received
0620 08:11:53.184431 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.184438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.184547 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.184736 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:53.185876 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.187282 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.188260 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.188871 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.188993 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.191578 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:53.191582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:53.191582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:53.191594 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:53.193704 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.194089 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.194092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.194093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.194209 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.194412 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.194413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.194413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.194733 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:53.194895 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.194966 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.195101 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260620 08:11:53.197302 26622 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:11:53.197769 26622 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:53.200929 26622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Generated new TSK 0
I20260620 08:11:53.201246 26622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:11:53.235509 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:53.321051 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:53.321257 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:53.321283 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:53.322878 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:53.322923 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:53.322950 26634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:11:53.324630 26634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:53.324870 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:53.325109 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:53.330267 26640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.330382 26639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.330260 26642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:53.330565 26634 server_base.cc:1034] running on GCE node
I20260620 08:11:53.330777 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:53.331064 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:53.332249 26634 hybrid_clock.cc:648] HybridClock initialized: now 1781943113332252 us; error 46 us; skew 500 ppm
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:53.334848 26634 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:11:53.335299 26634 webserver.cc:460] Webserver started at http://127.12.132.193:41581/ using document root <none> and password file <none>
I20260620 08:11:53.335518 26634 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:53.335565 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:53.335678 26634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:53.336786 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "7c545be9a3c9404ab391d642590978b1"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "678edf7627c8bdb04cd0787419e4510e"
server_key_iv: "f6559c1a36a92380b217cbef6be27356"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.337127 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "7c545be9a3c9404ab391d642590978b1"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "678edf7627c8bdb04cd0787419e4510e"
server_key_iv: "f6559c1a36a92380b217cbef6be27356"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.338517 26634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:53.339322 26649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.339649 26634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:53.339764 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "7c545be9a3c9404ab391d642590978b1"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "678edf7627c8bdb04cd0787419e4510e"
server_key_iv: "f6559c1a36a92380b217cbef6be27356"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.339849 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:53.346973 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:53.349217 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:53.349366 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:53.349579 26634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:53.349880 26634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:53.349931 26634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.349982 26634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:53.350018 26634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.356232 26634 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35829
I20260620 08:11:53.356438 26762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35829 every 8 connection(s)
I20260620 08:11:53.356668 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:11:53.360702 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26634
I20260620 08:11:53.360834 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:11:53.361011 12819 external_mini_cluster.cc:1428] Setting key 4da4f55c0de2979a66fa525e33ce7b24
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:53.366711 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.357416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:47323 (local address 127.12.132.254:43939)
0620 08:11:53.357461 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.357463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.357642 (+ 179us) server_negotiation.cc:407] Connection header received
0620 08:11:53.358171 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.358172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.358188 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.358257 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:53.359265 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.359798 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.360869 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.361083 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.361163 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.363306 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:53.363307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:53.363308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:53.363317 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:53.364794 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.365414 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.365418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.365419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.365547 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.365937 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.365938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.365939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.366155 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:53.366223 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.366581 (+ 358us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.366648 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:11:53.367023 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.357386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:47323)
0620 08:11:53.357527 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.357565 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.358075 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.358321 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.358325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.358939 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:53.359162 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.359167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.359893 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.359894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.360797 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.360800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.361175 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.361176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.361316 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.361856 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:53.361865 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:53.363180 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:53.364880 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.364886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.364889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.365226 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.365670 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.365671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.365672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.365815 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.366241 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:53.366249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:53.366424 (+ 175us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.366549 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.366732 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260620 08:11:53.367612 26763 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:53.367784 26763 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:53.368019 26763 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:53.368582 26572 ts_manager.cc:194] Registered new tserver with Master: 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
I20260620 08:11:53.369163 26572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:47323
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:11:53.389348 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:53.470175 26768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:53.470381 26768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:53.470404 26768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:53.471550 26768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:53.471597 26768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:53.471623 26768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:11:53.473073 26768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:53.473290 26768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:53.473526 26768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:53.478669 26774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.478718 26776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.478713 26773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:53.478942 26768 server_base.cc:1034] running on GCE node
I20260620 08:11:53.479167 26768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:53.479400 26768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:53.480580 26768 hybrid_clock.cc:648] HybridClock initialized: now 1781943113480570 us; error 36 us; skew 500 ppm
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:53.483065 26768 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:11:53.483520 26768 webserver.cc:460] Webserver started at http://127.12.132.194:36381/ using document root <none> and password file <none>
I20260620 08:11:53.483790 26768 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:53.483855 26768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:53.483995 26768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:53.485193 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "477514e939a348b0baf6d63744be38db"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "0002695eb8015b06b72ba9b9e1950b93"
server_key_iv: "3d1a059f132fd99f744eef8c89c1ab2d"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.485565 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "477514e939a348b0baf6d63744be38db"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "0002695eb8015b06b72ba9b9e1950b93"
server_key_iv: "3d1a059f132fd99f744eef8c89c1ab2d"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.487205 26768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:53.488341 26783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.488590 26768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:53.488680 26768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "477514e939a348b0baf6d63744be38db"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "0002695eb8015b06b72ba9b9e1950b93"
server_key_iv: "3d1a059f132fd99f744eef8c89c1ab2d"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.488744 26768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:53.499327 26768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:53.501631 26768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:53.501739 26768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:53.501917 26768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:53.502214 26768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:53.502274 26768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.502308 26768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:53.502338 26768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.508172 26768 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36923
I20260620 08:11:53.508215 26896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36923 every 8 connection(s)
I20260620 08:11:53.508566 26768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:11:53.514344 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26768
I20260620 08:11:53.514467 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:11:53.514650 12819 external_mini_cluster.cc:1428] Setting key 2a284374922b712c9d018393cbbf21b9
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:53.519996 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.509255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:46349 (local address 127.12.132.254:43939)
0620 08:11:53.509314 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.509317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.509499 (+ 182us) server_negotiation.cc:407] Connection header received
0620 08:11:53.510037 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.510039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.510051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.510117 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:53.511680 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.512205 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.513321 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.513537 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.513624 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.516438 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:53.516441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:53.516441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:53.516451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:53.518309 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.518753 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.518754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.518755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.518824 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.519216 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.519218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.519218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.519389 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:53.519465 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.519759 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.519870 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 08:11:53.520085 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.509237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:46349)
0620 08:11:53.509385 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.509419 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.509916 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.510186 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.510189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.511088 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:53.511395 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.511404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.512308 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.512310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.513240 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.513243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.513635 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.513636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.513754 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.514291 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:53.514299 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:53.516273 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:53.518407 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.518414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.518416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.518668 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.518950 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.518952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.518953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.519127 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.519482 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:53.519488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:53.519611 (+ 123us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.519710 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.519839 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260620 08:11:53.520752 26897 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:53.520870 26897 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:53.521047 26897 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:53.521466 26572 ts_manager.cc:194] Registered new tserver with Master: 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
I20260620 08:11:53.521873 26572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:46349
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:11:53.542919 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:53.624750 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:53.625052 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:53.625079 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:53.626139 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:53.626173 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:53.626199 26902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:11:53.627556 26902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:53.627733 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:53.627893 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:53.633311 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.633378 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:53.633378 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:53.633782 26902 server_base.cc:1034] running on GCE node
I20260620 08:11:53.634094 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:53.634402 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:53.635587 26902 hybrid_clock.cc:648] HybridClock initialized: now 1781943113635547 us; error 61 us; skew 500 ppm
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:53.638183 26902 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:11:53.638649 26902 webserver.cc:460] Webserver started at http://127.12.132.195:42173/ using document root <none> and password file <none>
I20260620 08:11:53.638885 26902 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:53.638950 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:53.639078 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:11:53.640077 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "03200cd83ced44cc87b5ecd156c73442"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "9d09bad4f2bb780f9415c36d9f3b3d49"
server_key_iv: "3290f6ac26c2c6253e61d9c50e06eec8"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.640424 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "03200cd83ced44cc87b5ecd156c73442"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "9d09bad4f2bb780f9415c36d9f3b3d49"
server_key_iv: "3290f6ac26c2c6253e61d9c50e06eec8"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.641754 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:53.642495 26917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.642710 26902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:53.642838 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "03200cd83ced44cc87b5ecd156c73442"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "9d09bad4f2bb780f9415c36d9f3b3d49"
server_key_iv: "3290f6ac26c2c6253e61d9c50e06eec8"
server_key_version: "encryptionkey@0"
I20260620 08:11:53.642900 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:53.666190 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:53.668185 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:53.668298 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:53.668483 26902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:53.668916 26902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:53.668972 26902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.669013 26902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:53.669039 26902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:53.674998 26902 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:37265
I20260620 08:11:53.675045 27030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:37265 every 8 connection(s)
I20260620 08:11:53.675341 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:11:53.678330 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 26902
I20260620 08:11:53.678503 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:11:53.678820 12819 external_mini_cluster.cc:1428] Setting key b72390fed8915225be3fe947b5111763
Jun 20 08:11:53 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:53.684959 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.675972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:47669 (local address 127.12.132.254:43939)
0620 08:11:53.676018 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.676020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.676250 (+ 230us) server_negotiation.cc:407] Connection header received
0620 08:11:53.676827 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.676828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.676839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.676911 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:53.677813 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.678311 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.679605 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.679789 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.679858 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.681888 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:53.681889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:53.681889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:53.681896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:53.683088 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.683766 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.683766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.683767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.683816 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.684300 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.684302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.684302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.684419 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:53.684485 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.684825 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.684893 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:11:53.685111 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.675977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:47669)
0620 08:11:53.676139 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.676172 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.676729 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.676982 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.676986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.677537 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:53.677719 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.677723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.678408 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.678409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.679518 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.679521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.679866 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.679867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.679994 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.680431 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:53.680439 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:53.681791 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:53.683227 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.683244 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.683246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.683686 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.683993 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:53.683994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:53.683994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:53.684165 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:53.684488 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:53.684533 (+ 45us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:53.684663 (+ 130us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.684753 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.684913 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260620 08:11:53.685530 27031 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:53.685652 27031 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:53.685818 27031 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:53.686383 26572 ts_manager.cc:194] Registered new tserver with Master: 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265)
I20260620 08:11:53.686834 26572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:47669
I20260620 08:11:53.691233 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:11:53.699657 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.693186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56984 (local address 127.12.132.254:43939)
0620 08:11:53.693296 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.693299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.693406 (+ 107us) server_negotiation.cc:407] Connection header received
0620 08:11:53.693430 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.693430 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.693442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.693531 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:53.694105 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.694572 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.695396 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.695674 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.695780 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.697037 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:53.697038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:53.697038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:53.697045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:53.698379 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.698728 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.698729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.698729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.698785 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:53.698978 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:53.698979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:53.698979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:53.699122 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:53.699207 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.699368 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.699452 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 08:11:53.701355 26572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:11:53.703120 26572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:11:53.709597 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.12.132.195:37265)
0620 08:11:53.705733 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.705738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.705750 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:11:53.705788 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.705790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.705807 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.705910 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:53.706091 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.706833 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.708360 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709099 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.709185 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.709239 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.709468 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.709508 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260620 08:11:53.709621 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37265 (local address 127.0.0.1:57336)
0620 08:11:53.705416 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.705458 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.705523 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.705916 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.705920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.705922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:53.706008 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.706010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.706940 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.706941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.708218 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.708226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709238 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.709239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.709326 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.709346 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.709441 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.709518 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":57,"threads_started":1}
I20260620 08:11:53.709942 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36923 (local address 127.0.0.1:43994)
0620 08:11:53.705734 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.705743 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.705797 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.706142 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.706143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.706145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:53.706200 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.706202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.707506 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.707507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.708552 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.708556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709649 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.709650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.709757 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.709760 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.709818 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.709872 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":79,"threads_started":1}
I20260620 08:11:53.710111 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.0.0.1:35504)
0620 08:11:53.705719 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.705731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.705801 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.706144 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.706145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.706146 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:53.706200 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.706203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.707487 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.707488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.708776 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.708779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709813 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.709814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.709891 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.709898 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.709932 (+ 34us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.710014 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":27,"thread_start_us":74,"threads_started":1}
I20260620 08:11:53.710242 27046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.12.132.194:36923)
0620 08:11:53.705727 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.705730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.705811 (+ 81us) server_negotiation.cc:407] Connection header received
0620 08:11:53.705864 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.705866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.705883 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.706029 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:53.706295 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.707381 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.708679 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709480 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.709552 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.709610 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.710100 (+ 490us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.710156 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260620 08:11:53.710525 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.705404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.12.132.193:35829)
0620 08:11:53.705672 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.705678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.705789 (+ 111us) server_negotiation.cc:407] Connection header received
0620 08:11:53.705908 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.705910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.705924 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.706072 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:53.706284 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.707381 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.708876 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.709582 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.710284 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.710349 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.710375 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.710417 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260620 08:11:53.710664 26965 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 (DEFAULT_TABLE table=test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:53.711158 26965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac84fefc3fd48ccb9b953868fb35135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:53.711292 26697 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 (DEFAULT_TABLE table=test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:53.711645 26697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac84fefc3fd48ccb9b953868fb35135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:53.714648 27049 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Bootstrap starting.
I20260620 08:11:53.714618 27048 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Bootstrap starting.
I20260620 08:11:53.715289 27049 tablet_bootstrap.cc:654] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:53.715178 26831 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 (DEFAULT_TABLE table=test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:53.715399 27048 tablet_bootstrap.cc:654] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:53.715556 26831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac84fefc3fd48ccb9b953868fb35135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:53.715847 27049 log.cc:824] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:53.715987 27048 log.cc:824] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:53.716748 27049 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: No bootstrap required, opened a new log
I20260620 08:11:53.716825 27049 ts_tablet_manager.cc:1397] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:53.716761 27048 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: No bootstrap required, opened a new log
I20260620 08:11:53.716903 27048 ts_tablet_manager.cc:1397] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:53.718266 27048 raft_consensus.cc:348] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.718423 27048 raft_consensus.cc:374] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:53.718467 27048 raft_consensus.cc:729] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03200cd83ced44cc87b5ecd156c73442, State: Initialized, Role: FOLLOWER
I20260620 08:11:53.718588 27048 consensus_queue.cc:260] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.718858 27048 ts_tablet_manager.cc:1428] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:53.718978 27031 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
I20260620 08:11:53.718861 27049 raft_consensus.cc:348] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.719020 27049 raft_consensus.cc:374] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:53.719072 27049 raft_consensus.cc:729] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c545be9a3c9404ab391d642590978b1, State: Initialized, Role: FOLLOWER
I20260620 08:11:53.719000 27053 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Bootstrap starting.
I20260620 08:11:53.719178 27049 consensus_queue.cc:260] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.719448 27049 ts_tablet_manager.cc:1428] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:11:53.719483 26763 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
I20260620 08:11:53.719895 27053 tablet_bootstrap.cc:654] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:53.720818 27053 log.cc:824] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:53.721588 27053 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: No bootstrap required, opened a new log
I20260620 08:11:53.721769 27053 ts_tablet_manager.cc:1397] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 08:11:53.723053 27053 raft_consensus.cc:348] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.723203 27053 raft_consensus.cc:374] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:53.723249 27053 raft_consensus.cc:729] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477514e939a348b0baf6d63744be38db, State: Initialized, Role: FOLLOWER
I20260620 08:11:53.723358 27053 consensus_queue.cc:260] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.723623 27053 ts_tablet_manager.cc:1428] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:11:53.723660 26897 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
W20260620 08:11:53.759176 26898 tablet.cc:2369] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:11:53.854418 27056 raft_consensus.cc:482] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:53.854557 27056 raft_consensus.cc:504] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.855017 27056 leader_election.cc:288] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
W20260620 08:11:53.857419 26764 tablet.cc:2369] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:11:53.857972 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.855302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:55531 (local address 127.12.132.193:35829)
0620 08:11:53.855377 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.855379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.855392 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:11:53.855469 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.855470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.855477 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.855551 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:53.855707 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.856182 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857136 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.857605 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857703 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.857712 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.857869 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.857910 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 08:11:53.857981 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.855263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.12.132.194:55531)
0620 08:11:53.855344 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.855355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.855405 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.855556 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.855557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.855559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:53.855633 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.855635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.856261 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.856262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.857050 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.857052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.857707 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.857794 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.857798 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.857849 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.857904 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 08:11:53.858006 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.855342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:33885 (local address 127.12.132.195:37265)
0620 08:11:53.855385 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.855387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.855515 (+ 128us) server_negotiation.cc:407] Connection header received
0620 08:11:53.855582 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.855583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.855591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.855672 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:53.855831 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.856288 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857204 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.857681 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857766 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.857777 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.857903 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.857943 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:11:53.858239 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.855277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37265 (local address 127.12.132.194:33885)
0620 08:11:53.855456 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.855465 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:53.855520 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:53.855678 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:53.855679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:53.855680 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:53.855762 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.855763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.856364 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.856365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.857116 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:53.857118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.857767 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.857768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:53.857818 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.857821 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:11:53.857890 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 08:11:53.857933 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260620 08:11:53.858403 26717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac84fefc3fd48ccb9b953868fb35135" candidate_uuid: "477514e939a348b0baf6d63744be38db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c545be9a3c9404ab391d642590978b1" is_pre_election: true
I20260620 08:11:53.858572 26717 raft_consensus.cc:2454] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477514e939a348b0baf6d63744be38db in term 0.
I20260620 08:11:53.858566 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac84fefc3fd48ccb9b953868fb35135" candidate_uuid: "477514e939a348b0baf6d63744be38db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03200cd83ced44cc87b5ecd156c73442" is_pre_election: true
I20260620 08:11:53.858700 26985 raft_consensus.cc:2454] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477514e939a348b0baf6d63744be38db in term 0.
I20260620 08:11:53.858814 26785 leader_election.cc:302] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477514e939a348b0baf6d63744be38db, 7c545be9a3c9404ab391d642590978b1; no voters:
I20260620 08:11:53.858937 27056 raft_consensus.cc:2781] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:11:53.858995 27056 raft_consensus.cc:482] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:11:53.859043 27056 raft_consensus.cc:3037] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:53.859763 27056 raft_consensus.cc:504] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.859907 27056 leader_election.cc:288] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [CANDIDATE]: Term 1 election: Requested vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
I20260620 08:11:53.860050 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac84fefc3fd48ccb9b953868fb35135" candidate_uuid: "477514e939a348b0baf6d63744be38db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03200cd83ced44cc87b5ecd156c73442"
I20260620 08:11:53.860049 26717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac84fefc3fd48ccb9b953868fb35135" candidate_uuid: "477514e939a348b0baf6d63744be38db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c545be9a3c9404ab391d642590978b1"
I20260620 08:11:53.860121 26985 raft_consensus.cc:3037] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:53.860131 26717 raft_consensus.cc:3037] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:53.860996 26985 raft_consensus.cc:2454] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477514e939a348b0baf6d63744be38db in term 1.
I20260620 08:11:53.860996 26717 raft_consensus.cc:2454] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477514e939a348b0baf6d63744be38db in term 1.
I20260620 08:11:53.861207 26784 leader_election.cc:302] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db; no voters:
I20260620 08:11:53.861286 27056 raft_consensus.cc:2781] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:53.861428 27056 raft_consensus.cc:686] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 1 LEADER]: Becoming Leader. State: Replica: 477514e939a348b0baf6d63744be38db, State: Running, Role: LEADER
I20260620 08:11:53.861543 27056 consensus_queue.cc:237] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:53.862324 26572 catalog_manager.cc:5495] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db reported cstate change: term changed from 0 to 1, leader changed from <none> to 477514e939a348b0baf6d63744be38db (127.12.132.194). New cstate: current_term: 1 leader_uuid: "477514e939a348b0baf6d63744be38db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } health_report { overall_health: UNKNOWN } } }
I20260620 08:11:53.892144 27046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:53.889568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44002 (local address 127.12.132.194:36923)
0620 08:11:53.889633 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:53.889635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:53.889645 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:11:53.889729 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:53.889730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:53.889748 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:53.889823 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:11:53.889967 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.890461 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.891245 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:53.891488 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:53.891563 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:53.891732 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:11:53.891879 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:11:53.891965 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:53.892015 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 08:11:53.892036 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 08:11:53.894542 26985 raft_consensus.cc:1261] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Refusing update from remote peer 477514e939a348b0baf6d63744be38db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:53.894563 26717 raft_consensus.cc:1261] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Refusing update from remote peer 477514e939a348b0baf6d63744be38db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:53.894935 27058 consensus_queue.cc:1035] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [LEADER]: Connected to new peer: Peer: permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:53.895071 27056 consensus_queue.cc:1035] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:53.897575 27061 mvcc.cc:205] Tried to move back new op lower bound from 7298838994509049856 to 7298838994377461760. Current Snapshot: MvccSnapshot[applied={T|T < 7298838994509049856}]
I20260620 08:11:53.897677 27062 mvcc.cc:205] Tried to move back new op lower bound from 7298838994509049856 to 7298838994377461760. Current Snapshot: MvccSnapshot[applied={T|T < 7298838994509049856}]
I20260620 08:11:53.900189 26572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:53.900301 26572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:53.901031 26572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Sending DeleteTablet for 3 replicas of tablet 5ac84fefc3fd48ccb9b953868fb35135
I20260620 08:11:53.901399 26965 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:53 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:57336
I20260620 08:11:53.901436 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26634
I20260620 08:11:53.901628 26697 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:53 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35504
I20260620 08:11:53.902227 27071 tablet_replica.cc:330] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: stopping tablet replica
I20260620 08:11:53.902231 26831 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:53 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:43994
I20260620 08:11:53.902336 27071 raft_consensus.cc:2229] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:53.902436 27071 raft_consensus.cc:2258] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:53.902702 27072 tablet_replica.cc:330] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: stopping tablet replica
I20260620 08:11:53.902879 27071 ts_tablet_manager.cc:1905] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:53.902908 27072 raft_consensus.cc:2229] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:11:53.903153 27072 raft_consensus.cc:2258] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:53.903726 27072 ts_tablet_manager.cc:1905] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:53.904943 27071 ts_tablet_manager.cc:1918] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:53.905035 27071 log.cc:1196] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/5ac84fefc3fd48ccb9b953868fb35135
I20260620 08:11:53.905298 27071 ts_tablet_manager.cc:1939] T 5ac84fefc3fd48ccb9b953868fb35135 P 03200cd83ced44cc87b5ecd156c73442: Deleting consensus metadata
I20260620 08:11:53.905714 26556 catalog_manager.cc:4841] TS 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): tablet 5ac84fefc3fd48ccb9b953868fb35135 (table test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]) successfully deleted
I20260620 08:11:53.906489 27072 ts_tablet_manager.cc:1918] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:53.906679 27072 log.cc:1196] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/5ac84fefc3fd48ccb9b953868fb35135
I20260620 08:11:53.907126 27072 ts_tablet_manager.cc:1939] T 5ac84fefc3fd48ccb9b953868fb35135 P 477514e939a348b0baf6d63744be38db: Deleting consensus metadata
I20260620 08:11:53.907732 26556 catalog_manager.cc:4841] TS 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): tablet 5ac84fefc3fd48ccb9b953868fb35135 (table test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]) successfully deleted
W20260620 08:11:53.908419 26557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:35829 (error 108)
W20260620 08:11:53.908957 27045 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:11:53.908812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.0.0.1:35516)
0620 08:11:53.908868 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:53.908903 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35829: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 08:11:53.909058 26557 catalog_manager.cc:4584] TS 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ac84fefc3fd48ccb9b953868fb35135: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35829: connect: Connection reset by peer (error 104)
I20260620 08:11:53.909337 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26768
I20260620 08:11:53.914194 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26902
I20260620 08:11:53.918746 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 26540
I20260620 08:11:53.924490 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--webserver_interface=127.12.132.254
--webserver_port=33123
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:54.012110 27074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:54.012364 27074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:54.012403 27074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:54.013546 27074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:54.013614 27074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:54.013640 27074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:54.013662 27074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:54.013688 27074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:54.015136 27074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=33123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:54.015415 27074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:54.015637 27074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:54.027283 27083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.027283 27080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.027372 27081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.027773 27074 server_base.cc:1034] running on GCE node
I20260620 08:11:54.028081 27074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:54.028406 27074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:54.029640 27074 hybrid_clock.cc:648] HybridClock initialized: now 1781943114029640 us; error 54 us; skew 500 ppm
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:54.037240 27074 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:54.037940 27074 webserver.cc:460] Webserver started at http://127.12.132.254:33123/ using document root <none> and password file <none>
I20260620 08:11:54.038303 27074 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:54.038532 27074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:54.040244 27074 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.041252 27090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.041574 27074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:54.041692 27074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "091671cfb3cb4096b1ba00aba8df4cd8"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "3d8d52633a509b253fe8ff08554ebff3"
server_key_iv: "60ad6ec65bcd4e8d721f4c43e6039294"
server_key_version: "encryptionkey@0"
I20260620 08:11:54.042130 27074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:54.090474 27074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:54.093609 27074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:54.093842 27074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:54.099646 27074 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:43939
I20260620 08:11:54.099673 27142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:43939 every 8 connection(s)
I20260620 08:11:54.100117 27074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:54.101087 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27074
I20260620 08:11:54.101394 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:35829
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=41581
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:54.102360 27143 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:11:54.103502 27143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap starting.
I20260620 08:11:54.115461 27143 log.cc:824] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:54.118007 27143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:11:54.118336 27143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap complete.
I20260620 08:11:54.121676 27143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.122014 27143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.122207 27143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.122300 27143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:54.122354 27143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:54.122422 27143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:11:54.123778 27143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.123898 27143 leader_election.cc:302] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091671cfb3cb4096b1ba00aba8df4cd8; no voters:
I20260620 08:11:54.124055 27143 leader_election.cc:288] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 08:11:54.124172 27147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:11:54.124317 27143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:54.124444 27147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 LEADER]: Becoming Leader. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Running, Role: LEADER
I20260620 08:11:54.124647 27147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.125135 27149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091671cfb3cb4096b1ba00aba8df4cd8. Latest consensus state: current_term: 2 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:54.125118 27148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:54.125253 27149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:54.125258 27148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:54.125594 27156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:54.126621 27156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:54.126725 27156 catalog_manager.cc:1259] Loaded cluster ID: fa1a47fa26574ed78b3db380fb4944f1
I20260620 08:11:54.126760 27156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:11:54.126981 27156 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 08:11:54.131682 27156 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 08:11:54.134781 27156 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:11:54.135102 27156 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Loaded TSK: 0
I20260620 08:11:54.135416 27156 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:11:54.195279 27145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:54.195500 27145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:54.195536 27145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:54.196661 27145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:54.196722 27145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:54.196768 27145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:11:54.198210 27145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:35829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=41581
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:54.198462 27145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:54.198681 27145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:54.204613 27171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.204763 27173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.204627 27170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.205050 27145 server_base.cc:1034] running on GCE node
I20260620 08:11:54.205264 27145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:54.205523 27145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:54.206704 27145 hybrid_clock.cc:648] HybridClock initialized: now 1781943114206652 us; error 87 us; skew 500 ppm
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:54.209157 27145 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:11:54.209594 27145 webserver.cc:460] Webserver started at http://127.12.132.193:41581/ using document root <none> and password file <none>
I20260620 08:11:54.209821 27145 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:54.209899 27145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:54.211364 27145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.212075 27180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.212235 27145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.212314 27145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "7c545be9a3c9404ab391d642590978b1"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "678edf7627c8bdb04cd0787419e4510e"
server_key_iv: "f6559c1a36a92380b217cbef6be27356"
server_key_version: "encryptionkey@0"
I20260620 08:11:54.212683 27145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:54.235672 27145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:54.238689 27145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:54.238885 27145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:54.239188 27145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:54.239743 27187 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 08:11:54.241621 27145 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 08:11:54.241720 27145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 08:11:54.241755 27145 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 08:11:54.242609 27145 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 08:11:54.242655 27145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:54.242717 27187 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Bootstrap starting.
I20260620 08:11:54.250675 27145 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35829
I20260620 08:11:54.250824 27294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35829 every 8 connection(s)
I20260620 08:11:54.251211 27145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:11:54.254513 27187 log.cc:824] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:54.257055 27187 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:11:54.257366 27187 tablet_bootstrap.cc:492] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Bootstrap complete.
I20260620 08:11:54.258103 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27145
I20260620 08:11:54.258406 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:36923
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=36381
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:54.258633 27187 ts_tablet_manager.cc:1397] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260620 08:11:54.259984 27187 raft_consensus.cc:348] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.260298 27187 raft_consensus.cc:729] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c545be9a3c9404ab391d642590978b1, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.260452 27187 consensus_queue.cc:260] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.260767 27187 ts_tablet_manager.cc:1428] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:54.267011 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.252350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36175 (local address 127.12.132.254:43939)
0620 08:11:54.252571 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.252580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.255114 (+ 2534us) server_negotiation.cc:407] Connection header received
0620 08:11:54.255912 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.255920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.255978 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.256114 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:54.257029 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.258445 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.259883 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.260449 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.260618 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.263102 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:54.263106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:54.263106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:54.263118 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:54.265138 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.265691 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.265693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.265694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.265777 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.266096 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.266097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.266097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.266326 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:54.266401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.266769 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.266874 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260620 08:11:54.267153 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.252298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:36175)
0620 08:11:54.254951 (+ 2653us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.254984 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.255683 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.256247 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.256251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.256631 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:54.256898 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.256904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.258561 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.258562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.259782 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.259785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.260705 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.260706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.260844 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.261434 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:54.261442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:54.263001 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:54.265246 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.265254 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.265256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.265596 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.265848 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.265849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.265849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.266000 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.266403 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:54.266410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:54.266581 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.266710 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.266896 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2616,"thread_start_us":106,"threads_started":1}
I20260620 08:11:54.267987 27295 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:54.268112 27295 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:54.268299 27295 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:54.269094 27107 ts_manager.cc:194] Registered new tserver with Master: 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
I20260620 08:11:54.270514 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:36175
I20260620 08:11:54.271384 27295 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:11:54.280059 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.269941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.0.0.1:35518)
0620 08:11:54.270164 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.270198 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.270267 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.270515 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.270520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.270888 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:54.270954 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.270956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.272344 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.272345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.272979 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.272982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.274155 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.274158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.274288 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.275141 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:54.275145 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:54.276639 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:54.278620 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.278627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.278628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.278820 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.279083 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.279084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.279084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.279146 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.279781 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:54.279787 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:54.279830 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.279900 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.279994 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":145,"threads_started":1}
I20260620 08:11:54.280161 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.269968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.12.132.193:35829)
0620 08:11:54.270380 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.270383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.270390 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:11:54.270416 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.270417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.270428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.270511 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:54.271033 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.272237 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.273123 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.273893 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.274060 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.276730 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:54.276738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:54.276739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:54.276748 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:54.278525 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.278901 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.278904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.278905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.278972 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.279275 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.279278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.279278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.279691 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:54.279789 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.279959 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.280064 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":386,"thread_start_us":94,"threads_started":1}
I20260620 08:11:54.280764 27215 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac84fefc3fd48ccb9b953868fb35135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:53 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35518
I20260620 08:11:54.281353 27304 tablet_replica.cc:330] stopping tablet replica
I20260620 08:11:54.281462 27304 raft_consensus.cc:2229] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:54.281539 27304 raft_consensus.cc:2258] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:54.281833 27304 ts_tablet_manager.cc:1905] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:54.283856 27304 ts_tablet_manager.cc:1918] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:54.283921 27304 log.cc:1196] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/5ac84fefc3fd48ccb9b953868fb35135
I20260620 08:11:54.284176 27304 ts_tablet_manager.cc:1939] T 5ac84fefc3fd48ccb9b953868fb35135 P 7c545be9a3c9404ab391d642590978b1: Deleting consensus metadata
I20260620 08:11:54.284716 27092 catalog_manager.cc:4841] TS 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829): tablet 5ac84fefc3fd48ccb9b953868fb35135 (table test-table [id=4c7f066cedc045e1815a4f9f15cbcc26]) successfully deleted
W20260620 08:11:54.352017 27300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:54.352250 27300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:54.352290 27300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:54.353433 27300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:54.353499 27300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:54.353545 27300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:11:54.354990 27300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:36923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=36381
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:54.355239 27300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:54.355470 27300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:54.360562 27310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.360623 27312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.360634 27300 server_base.cc:1034] running on GCE node
W20260620 08:11:54.360332 27309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.361159 27300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:54.361459 27300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:54.362629 27300 hybrid_clock.cc:648] HybridClock initialized: now 1781943114362616 us; error 48 us; skew 500 ppm
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:54.365581 27300 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:11:54.366074 27300 webserver.cc:460] Webserver started at http://127.12.132.194:36381/ using document root <none> and password file <none>
I20260620 08:11:54.366291 27300 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:54.366379 27300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:54.367883 27300 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.368631 27319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.368829 27300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:54.368939 27300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "477514e939a348b0baf6d63744be38db"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "0002695eb8015b06b72ba9b9e1950b93"
server_key_iv: "3d1a059f132fd99f744eef8c89c1ab2d"
server_key_version: "encryptionkey@0"
I20260620 08:11:54.369284 27300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:54.397907 27300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:54.400812 27300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:54.400961 27300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:54.401196 27300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:54.401523 27300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:54.401574 27300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.401610 27300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:54.401636 27300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.409003 27300 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36923
I20260620 08:11:54.409056 27432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36923 every 8 connection(s)
I20260620 08:11:54.409417 27300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:54.416550 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27300
I20260620 08:11:54.416823 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:37265
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=42173
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:54.419703 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.410299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:46191 (local address 127.12.132.254:43939)
0620 08:11:54.410366 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.410369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.410604 (+ 235us) server_negotiation.cc:407] Connection header received
0620 08:11:54.411076 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.411077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.411089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.411154 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:54.412081 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.412655 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.414070 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.414325 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.414418 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.416816 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:54.416818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:54.416818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:54.416831 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:54.418276 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.418691 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.418692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.418692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.418752 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.419005 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.419006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.419006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.419118 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:54.419191 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.419531 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.419617 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 08:11:54.419938 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.410259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:46191)
0620 08:11:54.410451 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.410498 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.410988 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.411204 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.411206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.411788 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:54.411983 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.411987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.412845 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.412846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.413876 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.413880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.414457 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.414458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.414589 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.415204 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:54.415214 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:54.416700 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:54.418379 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.418386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.418388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.418605 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.418820 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.418821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.418821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.418929 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.419184 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:54.419190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:54.419367 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.419480 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.419665 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":123,"threads_started":1}
I20260620 08:11:54.420665 27433 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:54.420778 27433 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:54.420966 27433 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:54.421386 27107 ts_manager.cc:194] Registered new tserver with Master: 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
I20260620 08:11:54.421941 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:46191
W20260620 08:11:54.500954 27436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:54.501147 27436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:54.501171 27436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:54.502364 27436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:54.502405 27436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:54.502431 27436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:11:54.503831 27436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:37265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=42173
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:54.504010 27436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:54.504176 27436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:54.509261 27441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.509353 27442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.509363 27444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.509490 27436 server_base.cc:1034] running on GCE node
I20260620 08:11:54.509670 27436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:54.509912 27436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:54.511065 27436 hybrid_clock.cc:648] HybridClock initialized: now 1781943114511050 us; error 25 us; skew 500 ppm
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:54.513724 27436 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:11:54.514197 27436 webserver.cc:460] Webserver started at http://127.12.132.195:42173/ using document root <none> and password file <none>
I20260620 08:11:54.514410 27436 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:54.514495 27436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:54.516042 27436 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.516713 27451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.516882 27436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:54.516986 27436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "03200cd83ced44cc87b5ecd156c73442"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "9d09bad4f2bb780f9415c36d9f3b3d49"
server_key_iv: "3290f6ac26c2c6253e61d9c50e06eec8"
server_key_version: "encryptionkey@0"
I20260620 08:11:54.517315 27436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:54.532137 27436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:54.534724 27436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:54.534843 27436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:54.535028 27436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:54.535355 27436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:54.535411 27436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.535445 27436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:54.535473 27436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.541221 27436 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:37265
I20260620 08:11:54.541288 27564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:37265 every 8 connection(s)
I20260620 08:11:54.541585 27436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:11:54.542624 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27436
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:11:54.550060 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.543021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.12.132.254:43939)
0620 08:11:54.543170 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.543172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.543182 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:11:54.543269 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.543270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.543281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.543341 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:54.543917 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.544435 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.545213 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.545494 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.545550 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.547112 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:54.547114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:54.547114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:54.547123 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:54.548878 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.549263 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.549264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.549265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.549327 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.549530 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.549531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.549531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.549646 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:54.549821 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.549857 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.549945 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260620 08:11:54.552373 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.542307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:49493 (local address 127.12.132.254:43939)
0620 08:11:54.542406 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.542408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.542813 (+ 405us) server_negotiation.cc:407] Connection header received
0620 08:11:54.543645 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.543647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.543666 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.543722 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:54.544910 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.545543 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.546991 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.547190 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.547293 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.549549 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:54.549550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:54.549550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:54.549557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:54.551073 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.551469 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.551470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.551470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.551529 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:54.551766 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:54.551767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:54.551767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:54.551875 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:54.551947 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.552207 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.552288 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 08:11:54.552608 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.542242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:49493)
0620 08:11:54.542499 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.542635 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.543515 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.543828 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.543833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.544558 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:54.544810 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.544815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.545634 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.545636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.546883 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.546890 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.547307 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.547309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.547481 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.547949 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:54.547958 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:54.549453 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:54.551160 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.551167 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.551169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.551383 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.551592 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:54.551593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:54.551594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:54.551698 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:54.551940 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:54.551947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:54.552078 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.552186 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.552344 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260620 08:11:54.553056 27565 heartbeater.cc:347] Connected to a master server at 127.12.132.254:43939
I20260620 08:11:54.553129 27565 heartbeater.cc:464] Registering TS with master...
I20260620 08:11:54.553273 27565 heartbeater.cc:511] Master 127.12.132.254:43939 requested a full tablet report, sending...
I20260620 08:11:54.553548 27107 ts_manager.cc:194] Registered new tserver with Master: 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265)
I20260620 08:11:54.553906 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:49493
I20260620 08:11:54.561976 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:11:54.565299 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.562518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.12.132.254:43939)
0620 08:11:54.562597 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.562599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.562607 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:11:54.562646 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.562647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.562680 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.562763 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:11:54.562911 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.563387 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.564458 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.564712 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.564803 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.564973 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:11:54.565074 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:11:54.565156 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.565194 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.565211 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 08:11:54.565788 27107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:11:54.568023 27107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:11:54.570417 27215 tablet_service.cc:1467] Processing CreateTablet for tablet af6a5657a801402094cb0962992cdfab (DEFAULT_TABLE table=test-table [id=815103d4765e4eec9c8139f446e26638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:54.570642 27215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6a5657a801402094cb0962992cdfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:54.572320 27187 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Bootstrap starting.
I20260620 08:11:54.572760 27187 tablet_bootstrap.cc:654] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:54.573346 27187 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: No bootstrap required, opened a new log
I20260620 08:11:54.573415 27187 ts_tablet_manager.cc:1397] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:54.573782 27187 raft_consensus.cc:348] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.573868 27187 raft_consensus.cc:374] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:54.573907 27187 raft_consensus.cc:729] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c545be9a3c9404ab391d642590978b1, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.573956 27187 consensus_queue.cc:260] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.574035 27187 ts_tablet_manager.cc:1428] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:54.574597 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.570279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36923 (local address 127.0.0.1:54194)
0620 08:11:54.570365 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.570374 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.570412 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.570860 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.570861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.570864 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:54.570917 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.570919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.572026 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.572027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.573489 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.573491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.574269 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.574270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.574382 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.574402 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.574461 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.574507 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":39,"thread_start_us":53,"threads_started":1}
I20260620 08:11:54.574592 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.570152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37265 (local address 127.0.0.1:52686)
0620 08:11:54.570234 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.570245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.570284 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.570637 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.570638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.570640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:54.570692 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.570693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.572080 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.572081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.573319 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.573322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.574336 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.574337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.574409 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.574414 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.574446 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.574498 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":21}
I20260620 08:11:54.574671 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.570485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.12.132.194:36923)
0620 08:11:54.570677 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.570686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.570721 (+ 35us) server_negotiation.cc:407] Connection header received
0620 08:11:54.570761 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.570763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.570778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.570848 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:54.570980 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.571897 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.573549 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.574197 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.574270 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.574330 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.574498 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.574565 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260620 08:11:54.574999 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.570249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.12.132.195:37265)
0620 08:11:54.570446 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.570450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.570463 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:11:54.570501 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.570503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.570520 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.570793 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:54.570849 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.571907 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.573404 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.574246 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.574773 (+ 527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.574842 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.574865 (+ 23us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.574905 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
I20260620 08:11:54.575410 27367 tablet_service.cc:1467] Processing CreateTablet for tablet af6a5657a801402094cb0962992cdfab (DEFAULT_TABLE table=test-table [id=815103d4765e4eec9c8139f446e26638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:54.575572 27499 tablet_service.cc:1467] Processing CreateTablet for tablet af6a5657a801402094cb0962992cdfab (DEFAULT_TABLE table=test-table [id=815103d4765e4eec9c8139f446e26638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:11:54.575726 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6a5657a801402094cb0962992cdfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:54.575882 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6a5657a801402094cb0962992cdfab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:11:54.578589 27575 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Bootstrap starting.
I20260620 08:11:54.579116 27575 tablet_bootstrap.cc:654] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:54.579617 27575 log.cc:824] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:54.579974 27576 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Bootstrap starting.
I20260620 08:11:54.580350 27575 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: No bootstrap required, opened a new log
I20260620 08:11:54.580437 27575 ts_tablet_manager.cc:1397] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:11:54.580534 27576 tablet_bootstrap.cc:654] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Neither blocks nor log segments found. Creating new log.
I20260620 08:11:54.580962 27576 log.cc:824] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:54.581583 27576 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: No bootstrap required, opened a new log
I20260620 08:11:54.581632 27576 ts_tablet_manager.cc:1397] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:11:54.581948 27575 raft_consensus.cc:348] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.582094 27575 raft_consensus.cc:374] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:54.582137 27575 raft_consensus.cc:729] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477514e939a348b0baf6d63744be38db, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.582276 27575 consensus_queue.cc:260] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.582584 27575 ts_tablet_manager.cc:1428] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:11:54.582605 27433 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
I20260620 08:11:54.583446 27576 raft_consensus.cc:348] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.583590 27576 raft_consensus.cc:374] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:11:54.583636 27576 raft_consensus.cc:729] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03200cd83ced44cc87b5ecd156c73442, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.583748 27576 consensus_queue.cc:260] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.584002 27576 ts_tablet_manager.cc:1428] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:11:54.584038 27565 heartbeater.cc:503] Master 127.12.132.254:43939 was elected leader, sending a full tablet report...
W20260620 08:11:54.660017 27434 tablet.cc:2369] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:11:54.694600 27580 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:54.694787 27580 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.695466 27580 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 477514e939a348b0baf6d63744be38db (127.12.132.194:36923), 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
I20260620 08:11:54.698894 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.695650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36923 (local address 127.12.132.195:60069)
0620 08:11:54.695697 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.695709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.695754 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.695956 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.695957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.695959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:54.696049 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.696051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.696787 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.696788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.697656 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.697659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.698575 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.698576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.698666 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.698671 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.698764 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.698811 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1}
I20260620 08:11:54.698939 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.695721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60069 (local address 127.12.132.194:36923)
0620 08:11:54.695768 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.695771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.695782 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:11:54.695816 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.695817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.695827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.695947 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:54.696139 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.696710 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.697790 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.698461 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.698547 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.698558 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.698785 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.698847 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:11:54.699478 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.695750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:41559 (local address 127.12.132.193:35829)
0620 08:11:54.695792 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.695794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.695991 (+ 197us) server_negotiation.cc:407] Connection header received
0620 08:11:54.696047 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.696048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.696076 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.696145 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:11:54.696334 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.697002 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.698432 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.699099 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.699160 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.699230 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.699386 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.699419 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:11:54.699397 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6a5657a801402094cb0962992cdfab" candidate_uuid: "03200cd83ced44cc87b5ecd156c73442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477514e939a348b0baf6d63744be38db" is_pre_election: true
I20260620 08:11:54.699544 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.695651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.12.132.195:41559)
0620 08:11:54.695910 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.695922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:54.695983 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:54.696164 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:54.696165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:54.696167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:11:54.696231 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.696233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.697163 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.697165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.698277 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:54.698281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.699198 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.699200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:54.699287 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.699293 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 08:11:54.699374 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:11:54.699429 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":80,"thread_start_us":106,"threads_started":1}
I20260620 08:11:54.699628 27387 raft_consensus.cc:2454] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03200cd83ced44cc87b5ecd156c73442 in term 0.
I20260620 08:11:54.699973 27249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6a5657a801402094cb0962992cdfab" candidate_uuid: "03200cd83ced44cc87b5ecd156c73442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c545be9a3c9404ab391d642590978b1" is_pre_election: true
I20260620 08:11:54.699949 27452 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db; no voters:
I20260620 08:11:54.700140 27249 raft_consensus.cc:2454] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03200cd83ced44cc87b5ecd156c73442 in term 0.
I20260620 08:11:54.700292 27580 raft_consensus.cc:2781] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:11:54.700351 27580 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:11:54.700384 27580 raft_consensus.cc:3037] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:54.701195 27580 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.701344 27580 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [CANDIDATE]: Term 1 election: Requested vote from peers 477514e939a348b0baf6d63744be38db (127.12.132.194:36923), 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829)
I20260620 08:11:54.701560 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6a5657a801402094cb0962992cdfab" candidate_uuid: "03200cd83ced44cc87b5ecd156c73442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477514e939a348b0baf6d63744be38db"
I20260620 08:11:54.701591 27249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6a5657a801402094cb0962992cdfab" candidate_uuid: "03200cd83ced44cc87b5ecd156c73442" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c545be9a3c9404ab391d642590978b1"
I20260620 08:11:54.701689 27249 raft_consensus.cc:3037] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:54.701654 27387 raft_consensus.cc:3037] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:11:54.702397 27387 raft_consensus.cc:2454] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03200cd83ced44cc87b5ecd156c73442 in term 1.
I20260620 08:11:54.702411 27249 raft_consensus.cc:2454] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03200cd83ced44cc87b5ecd156c73442 in term 1.
I20260620 08:11:54.702556 27452 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db; no voters:
I20260620 08:11:54.702653 27580 raft_consensus.cc:2781] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:11:54.702816 27580 raft_consensus.cc:686] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 1 LEADER]: Becoming Leader. State: Replica: 03200cd83ced44cc87b5ecd156c73442, State: Running, Role: LEADER
I20260620 08:11:54.702907 27580 consensus_queue.cc:237] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:54.703693 27107 catalog_manager.cc:5495] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "03200cd83ced44cc87b5ecd156c73442" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } health_report { overall_health: UNKNOWN } } }
I20260620 08:11:54.722628 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.720018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.12.132.195:37265)
0620 08:11:54.720101 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:54.720103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:54.720112 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:11:54.720146 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:54.720147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:54.720164 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:54.720234 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:11:54.720320 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.720840 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.721774 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:54.722019 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:54.722101 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:54.722261 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:11:54.722387 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:11:54.722481 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:54.722498 (+ 17us) server_negotiation.cc:299] Negotiation successful
0620 08:11:54.722523 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 08:11:54.725023 27249 raft_consensus.cc:1261] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Refusing update from remote peer 03200cd83ced44cc87b5ecd156c73442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:54.725183 27387 raft_consensus.cc:1261] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Refusing update from remote peer 03200cd83ced44cc87b5ecd156c73442: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:11:54.725441 27580 consensus_queue.cc:1035] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:54.725620 27582 consensus_queue.cc:1035] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:11:54.728569 27585 mvcc.cc:205] Tried to move back new op lower bound from 7298838997910196224 to 7298838997823737856. Current Snapshot: MvccSnapshot[applied={T|T < 7298838997910196224}]
I20260620 08:11:54.730048 27587 mvcc.cc:205] Tried to move back new op lower bound from 7298838997910196224 to 7298838997823737856. Current Snapshot: MvccSnapshot[applied={T|T < 7298838997910196224 or (T in {7298838997910196224})}]
I20260620 08:11:54.731153 27106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:54.731245 27106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:11:54.731956 27106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Sending DeleteTablet for 3 replicas of tablet af6a5657a801402094cb0962992cdfab
I20260620 08:11:54.732254 27215 tablet_service.cc:1514] Processing DeleteTablet for tablet af6a5657a801402094cb0962992cdfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:54 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35518
I20260620 08:11:54.732349 27304 tablet_replica.cc:330] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: stopping tablet replica
I20260620 08:11:54.732393 27304 raft_consensus.cc:2229] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:54.732365 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet af6a5657a801402094cb0962992cdfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:54 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:54194
I20260620 08:11:54.732575 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet af6a5657a801402094cb0962992cdfab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:11:54 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:52686
I20260620 08:11:54.732753 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27145
I20260620 08:11:54.732826 27596 tablet_replica.cc:330] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: stopping tablet replica
I20260620 08:11:54.733026 27596 raft_consensus.cc:2229] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:11:54.733170 27596 raft_consensus.cc:2258] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:54.733796 27595 tablet_replica.cc:330] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: stopping tablet replica
I20260620 08:11:54.733785 27596 ts_tablet_manager.cc:1905] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:54.733913 27595 raft_consensus.cc:2229] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:11:54.734031 27595 raft_consensus.cc:2258] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:11:54.735587 27596 ts_tablet_manager.cc:1918] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:54.735658 27596 log.cc:1196] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/af6a5657a801402094cb0962992cdfab
I20260620 08:11:54.735962 27596 ts_tablet_manager.cc:1939] T af6a5657a801402094cb0962992cdfab P 03200cd83ced44cc87b5ecd156c73442: Deleting consensus metadata
I20260620 08:11:54.736446 27091 catalog_manager.cc:4841] TS 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): tablet af6a5657a801402094cb0962992cdfab (table test-table [id=815103d4765e4eec9c8139f446e26638]) successfully deleted
I20260620 08:11:54.740936 27595 ts_tablet_manager.cc:1905] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 08:11:54.741928 27092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:35829 (error 108)
I20260620 08:11:54.742432 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.742315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:35829 (local address 127.0.0.1:35576)
0620 08:11:54.742364 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.742396 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 08:11:54.742482 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27300
W20260620 08:11:54.743664 27092 catalog_manager.cc:4584] TS 7c545be9a3c9404ab391d642590978b1 (127.12.132.193:35829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af6a5657a801402094cb0962992cdfab: Network error: Client connection negotiation failed: client connection to 127.12.132.193:35829: connect: Connection refused (error 111)
I20260620 08:11:54.748356 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27436
I20260620 08:11:54.748397 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:54.748284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36923 (local address 127.0.0.1:54196)
0620 08:11:54.748334 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:54.748366 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:36923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 08:11:54.752719 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27074
I20260620 08:11:54.758149 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--webserver_interface=127.12.132.254
--webserver_port=33123
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:43939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:54.839643 27598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:54.839855 27598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:54.839879 27598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:54.841418 27598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:11:54.841531 27598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:54.841552 27598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:54.841566 27598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:11:54.841581 27598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:11:54.843918 27598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:43939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:43939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=33123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:54.844327 27598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:54.844628 27598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:54.850401 27606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.850412 27604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:54.850550 27603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:54.850875 27598 server_base.cc:1034] running on GCE node
I20260620 08:11:54.851126 27598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:54.851423 27598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:54.852574 27598 hybrid_clock.cc:648] HybridClock initialized: now 1781943114852562 us; error 32 us; skew 500 ppm
Jun 20 08:11:54 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:54.854983 27598 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:11:54.855436 27598 webserver.cc:460] Webserver started at http://127.12.132.254:33123/ using document root <none> and password file <none>
I20260620 08:11:54.855613 27598 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:54.855667 27598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:54.857012 27598 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:54.857867 27613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:54.858110 27598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:11:54.858189 27598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "091671cfb3cb4096b1ba00aba8df4cd8"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "3d8d52633a509b253fe8ff08554ebff3"
server_key_iv: "60ad6ec65bcd4e8d721f4c43e6039294"
server_key_version: "encryptionkey@0"
I20260620 08:11:54.858520 27598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:54.875682 27598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:54.877907 27598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:54.878021 27598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:54.883256 27598 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:43939
I20260620 08:11:54.883298 27665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:43939 every 8 connection(s)
I20260620 08:11:54.883752 27598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:11:54.885418 27666 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:11:54.886108 27666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap starting.
I20260620 08:11:54.893306 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27598
I20260620 08:11:54.893626 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:35829
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=41581
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:54.895085 27666 log.cc:824] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:54.897513 27666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:11:54.897802 27666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8: Bootstrap complete.
I20260620 08:11:54.900200 27666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.900439 27666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Initialized, Role: FOLLOWER
I20260620 08:11:54.900648 27666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.900739 27666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:11:54.900792 27666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:11:54.900840 27666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 2 FOLLOWER]: Advancing to term 3
I20260620 08:11:54.901789 27666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.901908 27666 leader_election.cc:302] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091671cfb3cb4096b1ba00aba8df4cd8; no voters:
I20260620 08:11:54.902078 27666 leader_election.cc:288] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 08:11:54.902141 27670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 08:11:54.902287 27670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [term 3 LEADER]: Becoming Leader. State: Replica: 091671cfb3cb4096b1ba00aba8df4cd8, State: Running, Role: LEADER
I20260620 08:11:54.902333 27666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:11:54.902421 27670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } }
I20260620 08:11:54.902726 27670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091671cfb3cb4096b1ba00aba8df4cd8. Latest consensus state: current_term: 3 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:54.902830 27670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:54.902727 27671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091671cfb3cb4096b1ba00aba8df4cd8" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 43939 } } }
I20260620 08:11:54.903012 27671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091671cfb3cb4096b1ba00aba8df4cd8 [sys.catalog]: This master's current role is: LEADER
I20260620 08:11:54.903028 27678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:11:54.903941 27678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:11:54.904089 27678 catalog_manager.cc:1259] Loaded cluster ID: fa1a47fa26574ed78b3db380fb4944f1
I20260620 08:11:54.904304 27678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 08:11:54.904680 27678 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f837cd709ad google::LogMessage::Fail() at ??:0
@ 0x7f837cd7388a google::LogMessage::SendToLog() at ??:0
@ 0x7f837cd704de google::LogMessage::Flush() at ??:0
@ 0x7f837cd71a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8384c910f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f8384c9a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f837ec3e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f837ec33954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f837fa976db start_thread at ??:0
@ 0x7f837b86e61f clone at ??:0
W20260620 08:11:55.009999 27668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:55.010273 27668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:55.010313 27668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:55.012128 27668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:55.012187 27668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:55.012229 27668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:11:55.014429 27668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:35829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=41581
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:55.014673 27668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:55.014931 27668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:11:55.021759 27702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.021808 27703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.021808 27705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:55.022368 27668 server_base.cc:1034] running on GCE node
I20260620 08:11:55.022617 27668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:55.022912 27668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:55.024184 27668 hybrid_clock.cc:648] HybridClock initialized: now 1781943115024162 us; error 39 us; skew 500 ppm
Jun 20 08:11:55 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:55.027382 27668 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:11:55.027925 27668 webserver.cc:460] Webserver started at http://127.12.132.193:41581/ using document root <none> and password file <none>
I20260620 08:11:55.028182 27668 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:55.028250 27668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:55.029800 27668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:55.030532 27712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.030702 27668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:55.030818 27668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "7c545be9a3c9404ab391d642590978b1"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "678edf7627c8bdb04cd0787419e4510e"
server_key_iv: "f6559c1a36a92380b217cbef6be27356"
server_key_version: "encryptionkey@0"
I20260620 08:11:55.031168 27668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:55.068795 27668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:55.071529 27668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:55.071714 27668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:55.072016 27668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:55.072603 27719 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 08:11:55.074647 27668 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 08:11:55.074735 27668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 08:11:55.074790 27668 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 08:11:55.075434 27668 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 08:11:55.075491 27668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 08:11:55.075562 27719 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Bootstrap starting.
I20260620 08:11:55.082604 27668 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:35829
I20260620 08:11:55.082671 27826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:35829 every 8 connection(s)
I20260620 08:11:55.083102 27668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:11:55.086925 27719 log.cc:824] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:11:55.089890 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27668
I20260620 08:11:55.090154 27719 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:11:55.090299 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:36923
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=36381
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:11:55.090535 27719 tablet_bootstrap.cc:492] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Bootstrap complete.
I20260620 08:11:55.091766 27719 ts_tablet_manager.cc:1397] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260620 08:11:55.093219 27719 raft_consensus.cc:348] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:55.093482 27719 raft_consensus.cc:729] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c545be9a3c9404ab391d642590978b1, State: Initialized, Role: FOLLOWER
I20260620 08:11:55.093681 27719 consensus_queue.cc:260] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:55.093959 27719 ts_tablet_manager.cc:1428] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260620 08:11:55.144441 27829 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:11:55.083870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:45999)
0620 08:11:55.084034 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.084077 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:55.084646 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:55.144233 (+ 59587us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260620 08:11:55.144713 27827 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:43939 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:43939: Client connection negotiation failed: client connection to 127.12.132.254:43939: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:11:55.145218 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.144947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:42697)
0620 08:11:55.145138 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.145181 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260620 08:11:55.145823 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.145703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:46457)
0620 08:11:55.145767 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.145801 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 08:11:55.146042 27827 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:11:55.178023 27831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:55.178222 27831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:55.178247 27831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:55.179369 27831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:55.179416 27831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:55.179442 27831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:11:55.180891 27831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:36923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=36381
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:55.181128 27831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:55.181345 27831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:11:55.186681 27831 server_base.cc:1034] running on GCE node
W20260620 08:11:55.186681 27837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.186699 27838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.186726 27840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:55.187150 27831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:55.187388 27831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:55.188565 27831 hybrid_clock.cc:648] HybridClock initialized: now 1781943115188554 us; error 41 us; skew 500 ppm
Jun 20 08:11:55 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:55.191013 27831 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:11:55.191447 27831 webserver.cc:460] Webserver started at http://127.12.132.194:36381/ using document root <none> and password file <none>
I20260620 08:11:55.191757 27831 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:55.191814 27831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:55.193231 27831 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:55.193819 27847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.193964 27831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:11:55.194026 27831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "477514e939a348b0baf6d63744be38db"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "0002695eb8015b06b72ba9b9e1950b93"
server_key_iv: "3d1a059f132fd99f744eef8c89c1ab2d"
server_key_version: "encryptionkey@0"
I20260620 08:11:55.194326 27831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:55.215617 27831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:55.218079 27831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:55.218194 27831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:55.218381 27831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:55.218982 27854 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260620 08:11:55.220404 27854 ts_tablet_manager.cc:1852] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260620 08:11:55.220710 27854 ts_tablet_manager.cc:1905] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:11:55.222675 27854 ts_tablet_manager.cc:1918] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:11:55.222780 27854 log.cc:1196] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/af6a5657a801402094cb0962992cdfab
I20260620 08:11:55.223045 27854 ts_tablet_manager.cc:1939] T af6a5657a801402094cb0962992cdfab P 477514e939a348b0baf6d63744be38db: Deleting consensus metadata
I20260620 08:11:55.223249 27831 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260620 08:11:55.223301 27831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260620 08:11:55.223330 27831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:55.223341 27831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.229599 27831 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36923
I20260620 08:11:55.229665 27961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36923 every 8 connection(s)
I20260620 08:11:55.229949 27831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:11:55.231189 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.230559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:34507)
0620 08:11:55.230836 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.230975 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
W20260620 08:11:55.231469 27962 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:43939 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:43939: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
I20260620 08:11:55.231849 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.231744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:44627)
0620 08:11:55.231798 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.231831 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 08:11:55.232194 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.232127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:44241)
0620 08:11:55.232158 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.232179 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260620 08:11:55.232306 27962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 08:11:55.237411 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27831
I20260620 08:11:55.237663 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:37265
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:43939
--webserver_port=42173
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:11:55.326661 27965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:11:55.326906 27965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:11:55.326932 27965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:11:55.328119 27965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:11:55.328163 27965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:11:55.328193 27965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:11:55.329761 27965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:37265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=42173
--tserver_master_addrs=127.12.132.254:43939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:11:55.330024 27965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:11:55.330251 27965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:11:55.335636 27965 server_base.cc:1034] running on GCE node
W20260620 08:11:55.335603 27973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.335606 27970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:11:55.335590 27971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:11:55.335942 27965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:11:55.336215 27965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:11:55.337445 27965 hybrid_clock.cc:648] HybridClock initialized: now 1781943115337319 us; error 99 us; skew 500 ppm
Jun 20 08:11:55 dist-test-slave-z0sp krb5kdc[26527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:11:55.340718 27965 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:11:55.341415 27965 webserver.cc:460] Webserver started at http://127.12.132.195:42173/ using document root <none> and password file <none>
I20260620 08:11:55.341715 27965 fs_manager.cc:359] Metadata directory not provided
I20260620 08:11:55.341773 27965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:11:55.343468 27965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:11:55.344398 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.344630 27965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:11:55.344761 27965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "03200cd83ced44cc87b5ecd156c73442"
format_stamp: "Formatted at 2026-06-20 08:11:53 on dist-test-slave-z0sp"
server_key: "9d09bad4f2bb780f9415c36d9f3b3d49"
server_key_iv: "3290f6ac26c2c6253e61d9c50e06eec8"
server_key_version: "encryptionkey@0"
I20260620 08:11:55.345170 27965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:11:55.360021 27965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:11:55.362659 27965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:11:55.362888 27965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:11:55.363179 27965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:11:55.363574 27965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:11:55.363627 27965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.363667 27965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:11:55.363710 27965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:11:55.371467 27965 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:37265
I20260620 08:11:55.371563 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:37265 every 8 connection(s)
I20260620 08:11:55.371999 27965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:11:55.373284 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.372844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:44793)
0620 08:11:55.373021 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.373110 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260620 08:11:55.373575 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 27965
W20260620 08:11:55.373889 28094 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:43939 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:43939: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
I20260620 08:11:55.374274 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.374166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:50913)
0620 08:11:55.374221 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.374255 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 08:11:55.374596 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:55.374511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:48635)
0620 08:11:55.374551 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:55.374574 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 08:11:55.374727 28094 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 08:11:56.146899 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.146559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:36429)
0620 08:11:56.146758 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:56.146824 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
I20260620 08:11:56.232961 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.232745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:41359)
0620 08:11:56.232859 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:56.232911 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260620 08:11:56.375483 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.375159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:38827)
0620 08:11:56.375355 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:56.375407 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":84,"threads_started":1}
I20260620 08:11:56.743477 28101 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:56.743593 28101 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:56.743868 28101 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
Jun 20 08:11:56 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.194@KRBTEST.COM
Jun 20 08:11:56 dist-test-slave-z0sp krb5kdc[26527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943115, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.195@KRBTEST.COM
I20260620 08:11:56.755100 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.744196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36923 (local address 127.12.132.193:53957)
0620 08:11:56.744271 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:56.744279 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:56.744313 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:56.745178 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:56.745179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:56.745487 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:56.745671 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:56.745676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.746943 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.746944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:56.747811 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:56.747813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.748485 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.748486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:56.748622 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:56.749210 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:56.749218 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:56.750796 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:56.752999 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:56.753014 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:56.753017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:56.753407 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:56.753733 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:56.753734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:56.753735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:56.753890 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:56.754623 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:56.754632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:56.754805 (+ 173us) client_negotiation.cc:769] Sending connection context
0620 08:11:56.754907 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 08:11:56.755027 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":32,"thread_start_us":34,"threads_started":1}
I20260620 08:11:56.755606 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.744008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:37265 (local address 127.12.132.193:47379)
0620 08:11:56.744126 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:56.744137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:11:56.744192 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:11:56.745108 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:11:56.745111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:11:56.745485 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:11:56.745671 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:56.745685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.746872 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.746875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:56.747812 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:11:56.747815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.748507 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.748509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:11:56.748627 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:56.749267 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:11:56.749273 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:11:56.751189 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:11:56.754135 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:56.754136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:56.754136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:56.754305 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:56.754638 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:11:56.754639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:11:56.754639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:11:56.754706 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:11:56.755345 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:11:56.755345 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:11:56.755386 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 08:11:56.755469 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 08:11:56.755550 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":41,"thread_start_us":86,"threads_started":1}
I20260620 08:11:56.755590 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.744296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:53957 (local address 127.12.132.194:36923)
0620 08:11:56.744440 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:56.744448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:56.744466 (+ 18us) server_negotiation.cc:407] Connection header received
0620 08:11:56.744986 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:56.744988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:56.745048 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:56.745181 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:56.745777 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.746641 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.747920 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.748371 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.748532 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:56.750905 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:56.750909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:56.750909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:56.750918 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:56.752896 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:56.753507 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:56.753509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:56.753509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:56.753642 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:56.754044 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:56.754046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:56.754046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:56.754471 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:56.754666 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:56.755028 (+ 362us) server_negotiation.cc:299] Negotiation successful
0620 08:11:56.755394 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260620 08:11:56.755864 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:56.744104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:47379 (local address 127.12.132.195:37265)
0620 08:11:56.744338 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 08:11:56.744349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:11:56.744385 (+ 36us) server_negotiation.cc:407] Connection header received
0620 08:11:56.744894 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:11:56.744896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:11:56.744939 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:11:56.745054 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:11:56.745798 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.746645 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.747912 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:11:56.748379 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:11:56.748543 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:11:56.751441 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:11:56.751447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:11:56.751448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:11:56.751465 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:11:56.754040 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:56.754387 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:56.754390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:56.754391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:56.754502 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:11:56.754807 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:11:56.754808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:11:56.754808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:11:56.755229 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:11:56.755337 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 08:11:56.755547 (+ 210us) server_negotiation.cc:299] Negotiation successful
0620 08:11:56.755733 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
W20260620 08:11:56.756721 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
W20260620 08:11:56.756832 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
I20260620 08:11:56.756870 27713 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c545be9a3c9404ab391d642590978b1; no voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db
I20260620 08:11:56.756994 28101 raft_consensus.cc:2726] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:11:57.147980 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:57.147825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:35441)
0620 08:11:57.147893 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:57.147940 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 08:11:57.233659 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:57.233353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:46691)
0620 08:11:57.233548 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:57.233606 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260620 08:11:57.376567 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:57.376198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:59463)
0620 08:11:57.376409 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:57.376472 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":104,"threads_started":1}
I20260620 08:11:58.148667 28108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:58.148421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:45215)
0620 08:11:58.148586 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:58.148620 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":139,"threads_started":1}
I20260620 08:11:58.234588 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:58.234337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:40717)
0620 08:11:58.234515 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:58.234549 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260620 08:11:58.377281 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:58.377069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:35211)
0620 08:11:58.377203 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:58.377243 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260620 08:11:58.536244 28111 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:11:58.536337 28111 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:11:58.536600 28111 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
W20260620 08:11:58.537074 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
W20260620 08:11:58.537134 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
I20260620 08:11:58.537153 27713 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c545be9a3c9404ab391d642590978b1; no voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db
I20260620 08:11:58.537250 28111 raft_consensus.cc:2726] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:11:59.149617 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:59.149250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:43061)
0620 08:11:59.149463 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:59.149549 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260620 08:11:59.235386 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:59.235141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:50855)
0620 08:11:59.235300 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:59.235349 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260620 08:11:59.378248 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:11:59.377933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:32889)
0620 08:11:59.378127 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 08:11:59.378193 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":60,"threads_started":1}
I20260620 08:12:00.150396 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:00.150168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:56519)
0620 08:12:00.150304 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:00.150347 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":55,"threads_started":1}
I20260620 08:12:00.236281 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:00.236025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:56747)
0620 08:12:00.236200 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:00.236242 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260620 08:12:00.379067 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:00.378841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:54889)
0620 08:12:00.378984 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:00.379025 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260620 08:12:01.023298 28118 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:01.023392 28118 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:12:01.023515 28118 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
W20260620 08:12:01.024009 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
W20260620 08:12:01.024086 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
I20260620 08:12:01.024104 27713 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c545be9a3c9404ab391d642590978b1; no voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db
I20260620 08:12:01.024217 28118 raft_consensus.cc:2726] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:12:01.151598 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:01.151199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:54259)
0620 08:12:01.151443 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:01.151531 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":133,"threads_started":1}
I20260620 08:12:01.237304 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:01.236976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:47489)
0620 08:12:01.237220 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:01.237262 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":166,"threads_started":1}
I20260620 08:12:01.379851 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:01.379618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:53165)
0620 08:12:01.379775 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:01.379809 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":152,"threads_started":1}
I20260620 08:12:02.152649 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:02.152277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:46229)
0620 08:12:02.152482 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:02.152578 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":124,"threads_started":1}
I20260620 08:12:02.238376 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:02.237965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:57087)
0620 08:12:02.238184 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:02.238260 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260620 08:12:02.380599 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:02.380338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:44871)
0620 08:12:02.380512 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:02.380553 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260620 08:12:03.067049 28125 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:03.067181 28125 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:12:03.067426 28125 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
W20260620 08:12:03.067979 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
W20260620 08:12:03.068117 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
I20260620 08:12:03.068162 27713 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c545be9a3c9404ab391d642590978b1; no voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db
I20260620 08:12:03.068298 28125 raft_consensus.cc:2726] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:12:03.153375 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:03.153150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:34639)
0620 08:12:03.153283 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:03.153328 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260620 08:12:03.239302 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:03.239093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:50885)
0620 08:12:03.239225 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:03.239258 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260620 08:12:03.381321 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:03.381107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:56687)
0620 08:12:03.381250 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:03.381283 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260620 08:12:04.154423 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:04.154074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:48047)
0620 08:12:04.154319 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:04.154361 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":164,"threads_started":1}
I20260620 08:12:04.240255 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:04.240009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:56419)
0620 08:12:04.240181 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:04.240213 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260620 08:12:04.382326 28131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:04.382034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:53565)
0620 08:12:04.382233 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:04.382282 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260620 08:12:04.629383 28132 raft_consensus.cc:482] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:04.629539 28132 raft_consensus.cc:504] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03200cd83ced44cc87b5ecd156c73442" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 37265 } } peers { permanent_uuid: "477514e939a348b0baf6d63744be38db" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36923 } } peers { permanent_uuid: "7c545be9a3c9404ab391d642590978b1" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 35829 } }
I20260620 08:12:04.629829 28132 leader_election.cc:288] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265), 477514e939a348b0baf6d63744be38db (127.12.132.194:36923)
W20260620 08:12:04.630407 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03200cd83ced44cc87b5ecd156c73442 (127.12.132.195:37265): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
W20260620 08:12:04.630539 27713 leader_election.cc:341] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 477514e939a348b0baf6d63744be38db (127.12.132.194:36923): Not found: Tablet not found: af6a5657a801402094cb0962992cdfab
I20260620 08:12:04.630580 27713 leader_election.cc:302] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c545be9a3c9404ab391d642590978b1; no voters: 03200cd83ced44cc87b5ecd156c73442, 477514e939a348b0baf6d63744be38db
I20260620 08:12:04.630741 28132 raft_consensus.cc:2726] T af6a5657a801402094cb0962992cdfab P 7c545be9a3c9404ab391d642590978b1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 08:12:05.155505 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:05.155188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:54377)
0620 08:12:05.155373 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:05.155434 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
I20260620 08:12:05.241093 28134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:05.240819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:45209)
0620 08:12:05.240998 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:05.241040 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":61,"threads_started":1}
I20260620 08:12:05.383056 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:05.382820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:55493)
0620 08:12:05.382971 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:05.383009 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260620 08:12:06.156306 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:06.156083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:47069)
0620 08:12:06.156216 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:06.156257 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260620 08:12:06.241976 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:06.241608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:47831)
0620 08:12:06.241822 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:06.241907 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260620 08:12:06.383903 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:06.383619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:41975)
0620 08:12:06.383810 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:06.383851 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":65,"threads_started":1}
I20260620 08:12:07.157418 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:07.157025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:56203)
0620 08:12:07.157269 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:07.157348 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":169,"threads_started":1}
I20260620 08:12:07.242904 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:07.242651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:55763)
0620 08:12:07.242814 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:07.242856 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260620 08:12:07.384661 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:07.384378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:34305)
0620 08:12:07.384581 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:07.384620 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260620 08:12:08.158249 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:08.157979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:59583)
0620 08:12:08.158149 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:08.158188 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":65,"threads_started":1}
I20260620 08:12:08.243916 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:08.243592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:55545)
0620 08:12:08.243827 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:08.243872 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":112,"threads_started":1}
I20260620 08:12:08.385509 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:08.385266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:57177)
0620 08:12:08.385410 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:08.385463 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260620 08:12:09.159379 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:09.158970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:58555)
0620 08:12:09.159266 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:09.159318 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":166,"threads_started":1}
I20260620 08:12:09.244792 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:09.244455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:33619)
0620 08:12:09.244661 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:09.244726 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
I20260620 08:12:09.386312 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:09.386073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:56109)
0620 08:12:09.386228 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:09.386266 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260620 08:12:10.160229 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:10.160008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.193:51643)
0620 08:12:10.160145 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:10.160185 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260620 08:12:10.245704 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:10.245369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.194:52673)
0620 08:12:10.245571 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:10.245640 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":61,"threads_started":1}
I20260620 08:12:10.379994 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27668
I20260620 08:12:10.385092 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27831
I20260620 08:12:10.387557 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:10.387264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:43939 (local address 127.12.132.195:48643)
0620 08:12:10.387445 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:10.387495 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:43939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260620 08:12:10.390578 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 27965
2026-06-20T08:12:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18499 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:12:10 dist-test-slave-z0sp krb5kdc[28153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:12:10 dist-test-slave-z0sp krb5kdc[28153](info): set up 2 sockets
Jun 20 08:12:10 dist-test-slave-z0sp krb5kdc[28153](info): commencing operation
krb5kdc: starting...
W20260620 08:12:11.445354 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:12:11 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:12:11.577000 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:11.663396 28166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:11.663715 28166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:11.663754 28166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:11.664888 28166 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 08:12:11.664953 28166 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 08:12:11.664975 28166 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 08:12:11.665002 28166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:12:11.665033 28166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:11.665056 28166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:11.665086 28166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:12:11.665113 28166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:12:11.666564 28166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:44069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:11.666826 28166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:11.667040 28166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:11.672029 28171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:11.672258 28166 server_base.cc:1034] running on GCE node
W20260620 08:12:11.672039 28174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:11.672045 28172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:11.672629 28166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:11.672935 28166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:11.674136 28166 hybrid_clock.cc:648] HybridClock initialized: now 1781943131674121 us; error 39 us; skew 500 ppm
Jun 20 08:12:11 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943131, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:11.683405 28166 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:12:11.684145 28166 webserver.cc:460] Webserver started at http://127.12.132.254:42483/ using document root <none> and password file <none>
I20260620 08:12:11.684566 28166 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:11.684679 28166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:11.684893 28166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:12:11.686599 28166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "82fbd8ba7be64a28800fc9fc790ab025"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "6ca2d805cb11a211398268669b984914"
server_key_iv: "552d7fb8463535d178efcf641b95be42"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.687058 28166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "82fbd8ba7be64a28800fc9fc790ab025"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "6ca2d805cb11a211398268669b984914"
server_key_iv: "552d7fb8463535d178efcf641b95be42"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.689225 28166 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 08:12:11.690404 28181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:11.690649 28166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:11.690769 28166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "82fbd8ba7be64a28800fc9fc790ab025"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "6ca2d805cb11a211398268669b984914"
server_key_iv: "552d7fb8463535d178efcf641b95be42"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.690877 28166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:11.724684 28166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:11.727180 28166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:11.727347 28166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:11.732024 28166 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:44069
I20260620 08:12:11.732092 28233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:44069 every 8 connection(s)
I20260620 08:12:11.732461 28166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:12:11.733363 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28166
I20260620 08:12:11.733507 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:12:11.733649 28234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:11.733748 12819 external_mini_cluster.cc:1428] Setting key 4688f22fe13b883b13a8424cb1b2633e
I20260620 08:12:11.737759 28234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap starting.
I20260620 08:12:11.738413 28234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:11.738742 28234 log.cc:824] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:11.739496 28234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: No bootstrap required, opened a new log
Jun 20 08:12:11 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:11.741528 28234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:11.741679 28234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:11.741727 28234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Initialized, Role: FOLLOWER
I20260620 08:12:11.741890 28234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:11.741999 28234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:12:11.742039 28234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:12:11.742087 28234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:11.742792 28234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:11.742914 28234 leader_election.cc:302] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fbd8ba7be64a28800fc9fc790ab025; no voters:
I20260620 08:12:11.743085 28234 leader_election.cc:288] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:12:11.743151 28239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:12:11.743316 28239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 LEADER]: Becoming Leader. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Running, Role: LEADER
I20260620 08:12:11.743435 28234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:12:11.743542 28239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:11.744032 28240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:11.744127 28240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:11.744400 28241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fbd8ba7be64a28800fc9fc790ab025. Latest consensus state: current_term: 1 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:11.744567 28241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:11.744747 28247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:12:11.745361 28247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:12:11.746357 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:11.734326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.12.132.254:44069)
0620 08:12:11.734532 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:11.734544 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:11.734637 (+ 93us) server_negotiation.cc:407] Connection header received
0620 08:12:11.735395 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:11.735406 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:11.735477 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:11.735653 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:11.736447 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.737811 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.738740 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.739314 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.739424 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:11.741814 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:11.741817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:11.741817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:11.741828 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:11.744175 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:11.744621 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:11.744624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:11.744624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:11.744731 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:11.744978 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:11.744978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:11.744979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:11.745283 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:11.745901 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:11.745970 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 08:12:11.746127 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260620 08:12:11.747450 28247 catalog_manager.cc:1347] Generated new cluster ID: 33d058863b954f46bd8299fe734d8abe
I20260620 08:12:11.747504 28247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:12:11.764832 28247 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:12:11.765532 28247 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:12:11.772047 28247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Generated new TSK 0
I20260620 08:12:11.772221 28247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:12:11.808318 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:11.895437 28260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:11.895671 28260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:11.895710 28260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:11.896875 28260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:11.896944 28260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:11.896992 28260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:12:11.898486 28260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:11.898743 28260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:11.898985 28260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:11.904347 28265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:11.904392 28266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:11.904392 28268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:11.904646 28260 server_base.cc:1034] running on GCE node
I20260620 08:12:11.904829 28260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:11.905055 28260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:11.906252 28260 hybrid_clock.cc:648] HybridClock initialized: now 1781943131906218 us; error 52 us; skew 500 ppm
Jun 20 08:12:11 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943131, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:11.908823 28260 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:12:11.909273 28260 webserver.cc:460] Webserver started at http://127.12.132.193:39237/ using document root <none> and password file <none>
I20260620 08:12:11.909516 28260 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:11.909586 28260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:11.909952 28260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:12:11.911404 28260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "d8a067a0142846daa6640bc8c4d39c70"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "c357693767c72f21d8e56434dc1c2168"
server_key_iv: "ea51a54dd983af2ac5df393f78a887e0"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.911851 28260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "d8a067a0142846daa6640bc8c4d39c70"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "c357693767c72f21d8e56434dc1c2168"
server_key_iv: "ea51a54dd983af2ac5df393f78a887e0"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.913523 28260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 08:12:11.914446 28275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:11.914718 28260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:11.914827 28260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "d8a067a0142846daa6640bc8c4d39c70"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "c357693767c72f21d8e56434dc1c2168"
server_key_iv: "ea51a54dd983af2ac5df393f78a887e0"
server_key_version: "encryptionkey@0"
I20260620 08:12:11.914930 28260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:11.937309 28260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:11.940797 28260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:11.941016 28260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:11.941288 28260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:11.941725 28260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:11.941798 28260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:11.941838 28260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:11.941882 28260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:11.948261 28260 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:42105
I20260620 08:12:11.948328 28388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:42105 every 8 connection(s)
I20260620 08:12:11.948693 28260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:12:11.954010 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28260
I20260620 08:12:11.954164 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:12:11.954418 12819 external_mini_cluster.cc:1428] Setting key e97d431d4ded050bf2cf4e1ef6360b42
Jun 20 08:12:11 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943131, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:11.962481 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:11.949459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:42341)
0620 08:12:11.949757 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:11.949866 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:11.950613 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:11.951116 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:11.951120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:11.951764 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:11.952050 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:11.952056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.953149 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.953152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:11.954487 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:11.954494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.955048 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.955050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:11.955219 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:11.956215 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:11.956228 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:11.958370 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:11.960797 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:11.960805 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:11.960807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:11.961130 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:11.961443 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:11.961445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:11.961445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:11.961585 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:11.961929 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:11.961936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:11.962061 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 08:12:11.962153 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 08:12:11.962311 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":74,"threads_started":1}
I20260620 08:12:11.962497 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:11.949719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:42341 (local address 127.12.132.254:44069)
0620 08:12:11.949770 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:11.949773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:11.950079 (+ 306us) server_negotiation.cc:407] Connection header received
0620 08:12:11.950887 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:11.950888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:11.950899 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:11.950986 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:11.952358 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.952925 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.954625 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:11.954906 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:11.954987 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:11.958540 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:11.958543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:11.958543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:11.958552 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:11.960574 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:11.961227 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:11.961229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:11.961230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:11.961323 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:11.961666 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:11.961667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:11.961668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:11.961848 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:11.961931 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:11.962217 (+ 286us) server_negotiation.cc:299] Negotiation successful
0620 08:12:11.962340 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:12:11.963034 28389 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:11.963145 28389 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:11.963358 28389 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:11.963968 28198 ts_manager.cc:194] Registered new tserver with Master: d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:11.964656 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:42341
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:12:11.987823 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:12.070730 28394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:12.070957 28394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:12.070994 28394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:12.072095 28394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:12.072161 28394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:12.072206 28394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:12:12.073661 28394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:12.073930 28394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:12.074154 28394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:12.079341 28402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:12.079414 28400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:12.079341 28399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:12.079496 28394 server_base.cc:1034] running on GCE node
I20260620 08:12:12.079701 28394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:12.079901 28394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:12.081086 28394 hybrid_clock.cc:648] HybridClock initialized: now 1781943132081067 us; error 53 us; skew 500 ppm
Jun 20 08:12:12 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943132, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:12.083911 28394 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:12:12.084731 28394 webserver.cc:460] Webserver started at http://127.12.132.194:38105/ using document root <none> and password file <none>
I20260620 08:12:12.084991 28394 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:12.085059 28394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:12.085196 28394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:12:12.086239 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "510c411ebc1240b0a568f816d67f615c"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "73c386d206be58e19d0a53a0ed13e6f4"
server_key_iv: "73eb54b5ea2aec7f60486119ac0152b3"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.086596 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "510c411ebc1240b0a568f816d67f615c"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "73c386d206be58e19d0a53a0ed13e6f4"
server_key_iv: "73eb54b5ea2aec7f60486119ac0152b3"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.087980 28394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:12:12.088871 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.089157 28394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:12.089268 28394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "510c411ebc1240b0a568f816d67f615c"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "73c386d206be58e19d0a53a0ed13e6f4"
server_key_iv: "73eb54b5ea2aec7f60486119ac0152b3"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.089363 28394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:12.119076 28394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:12.121448 28394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:12.121608 28394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:12.121865 28394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:12.122190 28394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:12.122248 28394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.122293 28394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:12.122324 28394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.128620 28394 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36811
I20260620 08:12:12.128674 28522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36811 every 8 connection(s)
I20260620 08:12:12.128984 28394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:12:12.134500 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28394
I20260620 08:12:12.134696 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:12:12.134950 12819 external_mini_cluster.cc:1428] Setting key 59e9acf82c9472cbb720798ac739ccde
Jun 20 08:12:12 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943132, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:12.139647 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.129694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:44381 (local address 127.12.132.254:44069)
0620 08:12:12.129754 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.129756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.129992 (+ 236us) server_negotiation.cc:407] Connection header received
0620 08:12:12.130455 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.130456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.130466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.130527 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:12.131471 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.131997 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.133081 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.133286 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.133357 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.135658 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:12.135660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:12.135660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:12.135666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:12.137254 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.138095 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.138098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.138099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.138273 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.138928 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.138930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.138930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.139050 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:12.139113 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.139512 (+ 399us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.139583 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 08:12:12.139912 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.129660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:44381)
0620 08:12:12.129831 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.129871 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.130352 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.130610 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.130614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.131176 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:12.131374 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.131378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.132095 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.132096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.132990 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.132993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.133398 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.133399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.133516 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.133963 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:12.133971 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:12.135546 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:12.137369 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:12.137396 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:12.137398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:12.137825 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:12.138389 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:12.138390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:12.138391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:12.138704 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:12.139134 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:12.139142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:12.139310 (+ 168us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.139439 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.139629 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260620 08:12:12.140512 28523 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:12.140625 28523 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:12.140803 28523 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:12.141271 28198 ts_manager.cc:194] Registered new tserver with Master: 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811)
I20260620 08:12:12.141687 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:44381
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:12:12.163573 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:12.246557 28528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:12.246783 28528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:12.246821 28528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:12.247944 28528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:12.248008 28528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:12.248049 28528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:12:12.249562 28528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:12.250094 28528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:12.250438 28528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:12.256011 28536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:12.256011 28534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:12.256323 28528 server_base.cc:1034] running on GCE node
W20260620 08:12:12.256358 28533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:12.256613 28528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:12.256865 28528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:12.258046 28528 hybrid_clock.cc:648] HybridClock initialized: now 1781943132258048 us; error 43 us; skew 500 ppm
Jun 20 08:12:12 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943132, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:12.260784 28528 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:12:12.261273 28528 webserver.cc:460] Webserver started at http://127.12.132.195:39761/ using document root <none> and password file <none>
I20260620 08:12:12.261516 28528 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:12.261584 28528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:12.261718 28528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:12:12.262723 28528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "97d697ef8b8d4d488a98e073ea57ae03"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "188e2ed0c4c1332f8485751972a52d5b"
server_key_iv: "5f04cda9aa70578d92213dfd774331b4"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.263115 28528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "97d697ef8b8d4d488a98e073ea57ae03"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "188e2ed0c4c1332f8485751972a52d5b"
server_key_iv: "5f04cda9aa70578d92213dfd774331b4"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.264467 28528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:12:12.265336 28543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.265574 28528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:12.265677 28528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "97d697ef8b8d4d488a98e073ea57ae03"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "188e2ed0c4c1332f8485751972a52d5b"
server_key_iv: "5f04cda9aa70578d92213dfd774331b4"
server_key_version: "encryptionkey@0"
I20260620 08:12:12.265766 28528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:12.278296 28528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:12.280381 28528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:12.280583 28528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:12.280802 28528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:12.281101 28528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:12.281162 28528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.281203 28528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:12.281229 28528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:12.287714 28528 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:39877
I20260620 08:12:12.287832 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:39877 every 8 connection(s)
I20260620 08:12:12.288105 28528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:12:12.288599 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28528
I20260620 08:12:12.288715 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:12:12.288921 12819 external_mini_cluster.cc:1428] Setting key 32a404faeeeb1905aeaf5f33588f0771
Jun 20 08:12:12 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943132, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:12.298437 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.289211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:52127 (local address 127.12.132.254:44069)
0620 08:12:12.289255 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.289257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.289337 (+ 80us) server_negotiation.cc:407] Connection header received
0620 08:12:12.289984 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.289984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.289994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.290040 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:12.290916 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.291408 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.292392 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.292672 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.292753 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.295058 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:12.295061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:12.295062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:12.295075 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:12.296905 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.297332 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.297334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.297334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.297415 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.297747 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.297748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.297748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.297904 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:12.297984 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.298288 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.298372 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:12:12.298486 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.288846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:52127)
0620 08:12:12.289216 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.289262 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.289902 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.290088 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.290091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.290617 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:12.290807 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.290811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.291515 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.291515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.292301 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.292304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.292768 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.292783 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.292949 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.293348 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:12.293355 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:12.294801 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:12.296986 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:12.296992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:12.296993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:12.297214 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:12.297525 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:12.297525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:12.297526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:12.297635 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:12.298007 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:12.298013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:12.298124 (+ 111us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.298226 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.298337 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":86,"threads_started":1}
I20260620 08:12:12.298987 28657 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:12.299100 28657 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:12.299290 28657 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:12.299784 28198 ts_manager.cc:194] Registered new tserver with Master: 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877)
I20260620 08:12:12.300382 28198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:52127
I20260620 08:12:12.300812 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:12:12.308322 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.302247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.12.132.254:44069)
0620 08:12:12.302328 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.302329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.302337 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:12:12.302390 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.302390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.302398 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.302461 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:12.302988 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.303468 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.304241 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.304430 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.304486 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.305667 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:12.305668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:12.305668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:12.305674 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:12.307025 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.307413 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.307415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.307416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.307505 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:12.307816 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:12.307818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:12.307818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:12.307972 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:12.308068 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.308148 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.308207 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 08:12:12.310324 28197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:12:12.311908 28197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:12:12.319077 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.314750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:39877 (local address 127.0.0.1:54814)
0620 08:12:12.314845 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.314902 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.314963 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.315456 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.315459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.315461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:12.315532 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.315533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316885 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.316886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.317927 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.317929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318844 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.318898 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.318900 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.318959 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.318991 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":53,"threads_started":1}
I20260620 08:12:12.318995 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.314950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.0.0.1:35272)
0620 08:12:12.315236 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.315246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.315284 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.315532 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.315533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.315535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:12.315589 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.315590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316524 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.316525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.317781 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.317784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318681 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.318762 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.318775 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.318856 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.318909 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":76,"threads_started":1}
I20260620 08:12:12.319247 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.314804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36811 (local address 127.0.0.1:49540)
0620 08:12:12.315119 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.315130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.315185 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.315667 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.315667 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.315669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:12.315721 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.315722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316986 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.316987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.318095 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.318098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318994 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.319063 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.319067 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.319119 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.319162 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":93,"threads_started":1}
I20260620 08:12:12.319329 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.315066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.12.132.193:42105)
0620 08:12:12.315256 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.315261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.315278 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:12:12.315379 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.315381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.315397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.315462 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:12.315665 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316404 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.317864 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318585 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318680 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.318737 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.319038 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.319081 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260620 08:12:12.319509 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.314926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.12.132.194:36811)
0620 08:12:12.315426 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.315430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.315439 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:12:12.315475 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.315477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.315492 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.315592 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:12.315812 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316889 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318178 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318836 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318910 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.318958 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.319387 (+ 429us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.319433 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":104,"threads_started":1}
I20260620 08:12:12.319098 28673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.315082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.12.132.195:39877)
0620 08:12:12.315270 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.315273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.315281 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:12:12.315319 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.315321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.315337 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.315431 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:12.315617 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.316767 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318042 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.318755 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.318843 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.318897 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.318976 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.319015 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260620 08:12:12.320065 28323 tablet_service.cc:1467] Processing CreateTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 (DEFAULT_TABLE table=test-table [id=f4e41661a621471b9b421a4c5392afc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:12.320091 28591 tablet_service.cc:1467] Processing CreateTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 (DEFAULT_TABLE table=test-table [id=f4e41661a621471b9b421a4c5392afc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:12.320091 28457 tablet_service.cc:1467] Processing CreateTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 (DEFAULT_TABLE table=test-table [id=f4e41661a621471b9b421a4c5392afc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:12.320436 28323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11e5cb547c54afebc19ca7addcf2bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:12.320436 28591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11e5cb547c54afebc19ca7addcf2bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:12.320436 28457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f11e5cb547c54afebc19ca7addcf2bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:12.323330 28674 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Bootstrap starting.
I20260620 08:12:12.323880 28675 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Bootstrap starting.
I20260620 08:12:12.323907 28676 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Bootstrap starting.
I20260620 08:12:12.324110 28674 tablet_bootstrap.cc:654] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:12.324548 28675 tablet_bootstrap.cc:654] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:12.324548 28676 tablet_bootstrap.cc:654] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:12.324680 28674 log.cc:824] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:12.325178 28675 log.cc:824] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:12.325178 28676 log.cc:824] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:12.325491 28674 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: No bootstrap required, opened a new log
I20260620 08:12:12.325570 28674 ts_tablet_manager.cc:1397] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:12:12.326022 28675 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: No bootstrap required, opened a new log
I20260620 08:12:12.326097 28675 ts_tablet_manager.cc:1397] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:12:12.326231 28676 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: No bootstrap required, opened a new log
I20260620 08:12:12.326349 28676 ts_tablet_manager.cc:1397] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:12:12.327237 28674 raft_consensus.cc:348] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.327401 28674 raft_consensus.cc:374] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:12.327452 28674 raft_consensus.cc:729] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a067a0142846daa6640bc8c4d39c70, State: Initialized, Role: FOLLOWER
I20260620 08:12:12.327566 28674 consensus_queue.cc:260] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.327848 28674 ts_tablet_manager.cc:1428] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:12:12.327961 28389 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
I20260620 08:12:12.328035 28675 raft_consensus.cc:348] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.328199 28675 raft_consensus.cc:374] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:12.328231 28675 raft_consensus.cc:729] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d697ef8b8d4d488a98e073ea57ae03, State: Initialized, Role: FOLLOWER
I20260620 08:12:12.328351 28675 consensus_queue.cc:260] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.328390 28676 raft_consensus.cc:348] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.328570 28676 raft_consensus.cc:374] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:12.328619 28676 raft_consensus.cc:729] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510c411ebc1240b0a568f816d67f615c, State: Initialized, Role: FOLLOWER
I20260620 08:12:12.328673 28675 ts_tablet_manager.cc:1428] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:12:12.328722 28676 consensus_queue.cc:260] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.328788 28657 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
I20260620 08:12:12.328994 28676 ts_tablet_manager.cc:1428] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:12:12.329020 28523 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
W20260620 08:12:12.379604 28524 tablet.cc:2369] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:12:12.401516 28682 raft_consensus.cc:482] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:12.401818 28682 raft_consensus.cc:504] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.402208 28682 leader_election.cc:288] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877), d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:12.406142 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.402608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:49711 (local address 127.12.132.193:42105)
0620 08:12:12.402662 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.402664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.402782 (+ 118us) server_negotiation.cc:407] Connection header received
0620 08:12:12.402858 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.402858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.402867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.402936 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:12.403089 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.403836 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405108 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.405693 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405780 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.405788 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.406030 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.406071 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 08:12:12.406142 28673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.402522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:39751 (local address 127.12.132.195:39877)
0620 08:12:12.402610 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.402613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.402621 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:12:12.402653 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.402654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.402664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.402718 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:12.402901 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.403641 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405108 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.405699 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405780 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.405788 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.406030 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.406071 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 08:12:12.406159 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.402408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:39877 (local address 127.12.132.194:39751)
0620 08:12:12.402477 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.402491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.402552 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.402744 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.402745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.402747 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:12.402827 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.402828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.403740 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.403741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.404987 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.404990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.405838 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.405936 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.405944 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.406012 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.406072 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":32}
I20260620 08:12:12.406170 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.402536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.12.132.194:49711)
0620 08:12:12.402696 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.402705 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:12.402789 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:12.402940 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:12.402940 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:12.402942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:12.403017 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.403018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.403913 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.403913 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.405024 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:12.405025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.405838 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.405839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:12.405936 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.405944 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 08:12:12.406012 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:12:12.406069 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260620 08:12:12.406575 28611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11e5cb547c54afebc19ca7addcf2bf1" candidate_uuid: "510c411ebc1240b0a568f816d67f615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d697ef8b8d4d488a98e073ea57ae03" is_pre_election: true
I20260620 08:12:12.406991 28343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11e5cb547c54afebc19ca7addcf2bf1" candidate_uuid: "510c411ebc1240b0a568f816d67f615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a067a0142846daa6640bc8c4d39c70" is_pre_election: true
I20260620 08:12:12.407039 28611 raft_consensus.cc:2454] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 510c411ebc1240b0a568f816d67f615c in term 0.
I20260620 08:12:12.407150 28343 raft_consensus.cc:2454] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 510c411ebc1240b0a568f816d67f615c in term 0.
I20260620 08:12:12.407425 28411 leader_election.cc:302] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510c411ebc1240b0a568f816d67f615c, 97d697ef8b8d4d488a98e073ea57ae03; no voters:
I20260620 08:12:12.407722 28682 raft_consensus.cc:2781] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:12:12.407795 28682 raft_consensus.cc:482] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:12:12.407829 28682 raft_consensus.cc:3037] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:12.408764 28682 raft_consensus.cc:504] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.408934 28682 leader_election.cc:288] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [CANDIDATE]: Term 1 election: Requested vote from peers 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877), d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:12.409111 28343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11e5cb547c54afebc19ca7addcf2bf1" candidate_uuid: "510c411ebc1240b0a568f816d67f615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a067a0142846daa6640bc8c4d39c70"
I20260620 08:12:12.409111 28611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f11e5cb547c54afebc19ca7addcf2bf1" candidate_uuid: "510c411ebc1240b0a568f816d67f615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d697ef8b8d4d488a98e073ea57ae03"
I20260620 08:12:12.409197 28343 raft_consensus.cc:3037] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:12.409221 28611 raft_consensus.cc:3037] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:12.410099 28611 raft_consensus.cc:2454] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 510c411ebc1240b0a568f816d67f615c in term 1.
I20260620 08:12:12.410099 28343 raft_consensus.cc:2454] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 510c411ebc1240b0a568f816d67f615c in term 1.
I20260620 08:12:12.410321 28411 leader_election.cc:302] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510c411ebc1240b0a568f816d67f615c, 97d697ef8b8d4d488a98e073ea57ae03; no voters:
I20260620 08:12:12.410429 28682 raft_consensus.cc:2781] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:12:12.410607 28682 raft_consensus.cc:686] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 1 LEADER]: Becoming Leader. State: Replica: 510c411ebc1240b0a568f816d67f615c, State: Running, Role: LEADER
I20260620 08:12:12.410734 28682 consensus_queue.cc:237] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:12.411520 28198 catalog_manager.cc:5495] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c reported cstate change: term changed from 0 to 1, leader changed from <none> to 510c411ebc1240b0a568f816d67f615c (127.12.132.194). New cstate: current_term: 1 leader_uuid: "510c411ebc1240b0a568f816d67f615c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } health_report { overall_health: UNKNOWN } } }
I20260620 08:12:12.436003 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.433523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.12.132.194:36811)
0620 08:12:12.433620 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:12.433622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:12.433631 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:12:12.433669 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:12.433670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:12.433688 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:12.433764 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:12:12.433939 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.434445 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.435201 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:12.435408 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:12.435481 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:12.435668 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:12:12.435770 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:12:12.435874 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:12.435894 (+ 20us) server_negotiation.cc:299] Negotiation successful
0620 08:12:12.435913 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 08:12:12.438369 28343 raft_consensus.cc:1261] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Refusing update from remote peer 510c411ebc1240b0a568f816d67f615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:12:12.438556 28611 raft_consensus.cc:1261] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Refusing update from remote peer 510c411ebc1240b0a568f816d67f615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:12:12.438762 28682 consensus_queue.cc:1035] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:12:12.438951 28684 consensus_queue.cc:1035] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:12:12.441542 28688 mvcc.cc:205] Tried to move back new op lower bound from 7298839070463537152 to 7298839070354993152. Current Snapshot: MvccSnapshot[applied={T|T < 7298839070463537152}]
I20260620 08:12:12.442238 28689 mvcc.cc:205] Tried to move back new op lower bound from 7298839070463537152 to 7298839070354993152. Current Snapshot: MvccSnapshot[applied={T|T < 7298839070463537152 or (T in {7298839070463537152})}]
I20260620 08:12:12.444839 28197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:12:12.444932 28197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:12:12.445737 28197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Sending DeleteTablet for 3 replicas of tablet f11e5cb547c54afebc19ca7addcf2bf1
I20260620 08:12:12.446033 28457 tablet_service.cc:1514] Processing DeleteTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:12 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:49540
I20260620 08:12:12.446095 28323 tablet_service.cc:1514] Processing DeleteTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:12 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:35272
I20260620 08:12:12.446278 28697 tablet_replica.cc:330] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: stopping tablet replica
I20260620 08:12:12.446286 28591 tablet_service.cc:1514] Processing DeleteTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:12 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:54814
I20260620 08:12:12.446448 28697 raft_consensus.cc:2229] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:12:12.446489 28699 tablet_replica.cc:330] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: stopping tablet replica
I20260620 08:12:12.446503 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28260
I20260620 08:12:12.446585 28699 raft_consensus.cc:2229] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:12:12.446691 28699 raft_consensus.cc:2258] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:12:12.446700 28697 raft_consensus.cc:2258] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:12:12.447139 28699 ts_tablet_manager.cc:1905] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:12:12.447299 28697 ts_tablet_manager.cc:1905] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:12:12.449033 28697 ts_tablet_manager.cc:1918] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:12:12.449170 28697 log.cc:1196] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/f11e5cb547c54afebc19ca7addcf2bf1
I20260620 08:12:12.449167 28699 ts_tablet_manager.cc:1918] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:12:12.449230 28699 log.cc:1196] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/f11e5cb547c54afebc19ca7addcf2bf1
I20260620 08:12:12.449623 28699 ts_tablet_manager.cc:1939] T f11e5cb547c54afebc19ca7addcf2bf1 P 97d697ef8b8d4d488a98e073ea57ae03: Deleting consensus metadata
I20260620 08:12:12.449635 28697 ts_tablet_manager.cc:1939] T f11e5cb547c54afebc19ca7addcf2bf1 P 510c411ebc1240b0a568f816d67f615c: Deleting consensus metadata
I20260620 08:12:12.450673 28182 catalog_manager.cc:4841] TS 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811): tablet f11e5cb547c54afebc19ca7addcf2bf1 (table test-table [id=f4e41661a621471b9b421a4c5392afc5]) successfully deleted
I20260620 08:12:12.450768 28183 catalog_manager.cc:4841] TS 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877): tablet f11e5cb547c54afebc19ca7addcf2bf1 (table test-table [id=f4e41661a621471b9b421a4c5392afc5]) successfully deleted
W20260620 08:12:12.454133 28183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:42105 (error 108)
I20260620 08:12:12.454690 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:12.454553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.0.0.1:35274)
0620 08:12:12.454614 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:12.454652 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:42105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 08:12:12.454739 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28394
W20260620 08:12:12.454850 28183 catalog_manager.cc:4584] TS d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f11e5cb547c54afebc19ca7addcf2bf1: Network error: Client connection negotiation failed: client connection to 127.12.132.193:42105: connect: Connection refused (error 111)
I20260620 08:12:12.459723 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28528
I20260620 08:12:12.464437 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28166
I20260620 08:12:32.470376 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--webserver_interface=127.12.132.254
--webserver_port=42483
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:32.552848 28701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:32.553045 28701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:32.553066 28701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:32.554126 28701 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 08:12:32.554157 28701 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 08:12:32.554169 28701 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 08:12:32.554180 28701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:12:32.554193 28701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:32.554205 28701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:32.554217 28701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:12:32.554229 28701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:12:32.555588 28701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:44069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:32.555780 28701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:32.556018 28701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:12:32.564975 28701 server_base.cc:1034] running on GCE node
W20260620 08:12:32.565080 28708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:32.565080 28707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:32.565016 28710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:32.565544 28701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:32.565881 28701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:32.567036 28701 hybrid_clock.cc:648] HybridClock initialized: now 1781943152567019 us; error 41 us; skew 500 ppm
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:32.574627 28701 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:12:32.575268 28701 webserver.cc:460] Webserver started at http://127.12.132.254:42483/ using document root <none> and password file <none>
I20260620 08:12:32.575573 28701 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:32.575690 28701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:32.577414 28701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:32.578251 28717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:32.578516 28701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:32.578631 28701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "82fbd8ba7be64a28800fc9fc790ab025"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "6ca2d805cb11a211398268669b984914"
server_key_iv: "552d7fb8463535d178efcf641b95be42"
server_key_version: "encryptionkey@0"
I20260620 08:12:32.579047 28701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:32.605906 28701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:32.608049 28701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:32.608227 28701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:32.612658 28701 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:44069
I20260620 08:12:32.612694 28769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:44069 every 8 connection(s)
I20260620 08:12:32.613116 28701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:12:32.614580 28770 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:12:32.615338 28770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap starting.
I20260620 08:12:32.616978 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28701
I20260620 08:12:32.617265 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:42105
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=39237
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:12:32.624852 28770 log.cc:824] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:32.627169 28770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:32.627491 28770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap complete.
I20260620 08:12:32.629858 28770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:32.630079 28770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Initialized, Role: FOLLOWER
I20260620 08:12:32.630223 28770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:32.630326 28770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:12:32.630385 28770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:12:32.630455 28770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:12:32.631551 28770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:32.631672 28770 leader_election.cc:302] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fbd8ba7be64a28800fc9fc790ab025; no voters:
I20260620 08:12:32.631899 28770 leader_election.cc:288] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 08:12:32.631933 28774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:12:32.632045 28774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 LEADER]: Becoming Leader. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Running, Role: LEADER
I20260620 08:12:32.632170 28770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:12:32.632242 28774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:32.632709 28775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:32.633054 28775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:32.632851 28776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fbd8ba7be64a28800fc9fc790ab025. Latest consensus state: current_term: 2 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:32.633285 28776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:32.633334 28783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:12:32.634212 28783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:12:32.634322 28783 catalog_manager.cc:1259] Loaded cluster ID: 33d058863b954f46bd8299fe734d8abe
I20260620 08:12:32.634361 28783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:12:32.635535 28783 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:12:32.635713 28783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Loaded TSK: 0
I20260620 08:12:32.648918 28783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Generated new TSK 1
I20260620 08:12:32.649009 28783 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:12:32.700084 28772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:32.700307 28772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:32.700345 28772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:32.701723 28772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:32.701779 28772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:32.701822 28772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:12:32.703368 28772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:42105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=39237
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:32.703629 28772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:32.703861 28772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:32.708976 28798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:32.708961 28797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:32.709125 28772 server_base.cc:1034] running on GCE node
W20260620 08:12:32.708981 28800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:32.709458 28772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:32.709707 28772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:32.710929 28772 hybrid_clock.cc:648] HybridClock initialized: now 1781943152710915 us; error 31 us; skew 500 ppm
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:32.713994 28772 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:12:32.714524 28772 webserver.cc:460] Webserver started at http://127.12.132.193:39237/ using document root <none> and password file <none>
I20260620 08:12:32.714787 28772 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:32.714874 28772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:32.716635 28772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:12:32.717643 28807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:32.717928 28772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:12:32.718149 28772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "d8a067a0142846daa6640bc8c4d39c70"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "c357693767c72f21d8e56434dc1c2168"
server_key_iv: "ea51a54dd983af2ac5df393f78a887e0"
server_key_version: "encryptionkey@0"
I20260620 08:12:32.718530 28772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:32.751858 28772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:32.754398 28772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:32.754560 28772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:32.754822 28772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:32.755280 28814 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 08:12:32.757094 28772 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 08:12:32.757181 28772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 08:12:32.757231 28772 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 08:12:32.757858 28772 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 08:12:32.757916 28772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 08:12:32.757980 28814 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Bootstrap starting.
I20260620 08:12:32.765022 28772 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:42105
I20260620 08:12:32.765115 28921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:42105 every 8 connection(s)
I20260620 08:12:32.765413 28772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:12:32.769764 28814 log.cc:824] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:32.772289 28814 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:32.772650 28814 tablet_bootstrap.cc:492] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Bootstrap complete.
I20260620 08:12:32.772902 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28772
I20260620 08:12:32.773335 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:36811
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=38105
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:32.773937 28814 ts_tablet_manager.cc:1397] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.012s
I20260620 08:12:32.775368 28814 raft_consensus.cc:348] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:32.775749 28814 raft_consensus.cc:729] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a067a0142846daa6640bc8c4d39c70, State: Initialized, Role: FOLLOWER
I20260620 08:12:32.775916 28814 consensus_queue.cc:260] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:32.776213 28814 ts_tablet_manager.cc:1428] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:12:32.777673 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.766244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:38625 (local address 127.12.132.254:44069)
0620 08:12:32.766427 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:32.766435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:32.766495 (+ 60us) server_negotiation.cc:407] Connection header received
0620 08:12:32.767345 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:32.767351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:32.767394 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:32.767506 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:32.768487 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.769405 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.770833 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.771222 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.771301 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.774016 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:32.774020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:32.774020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:32.774033 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:32.775967 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.776448 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.776450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.776451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.776594 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.776869 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.776870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.776870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.777061 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:32.777123 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:32.777449 (+ 326us) server_negotiation.cc:299] Negotiation successful
0620 08:12:32.777544 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":118,"threads_started":1}
I20260620 08:12:32.777837 28924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.766148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:38625)
0620 08:12:32.766364 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:32.766408 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:32.767116 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:32.767581 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:32.767586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:32.768033 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:32.768353 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.768362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.769531 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.769531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.770742 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.770745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.771331 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.771332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.771475 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.772008 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:32.772017 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:32.773903 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:32.776071 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.776078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.776079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.776363 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.776652 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.776653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.776653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.776798 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.777131 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:32.777138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:32.777293 (+ 155us) client_negotiation.cc:769] Sending connection context
0620 08:12:32.777410 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 08:12:32.777597 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":134,"threads_started":1}
I20260620 08:12:32.778606 28922 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:32.778704 28922 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:32.778887 28922 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:32.779565 28734 ts_manager.cc:194] Registered new tserver with Master: d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:32.780653 28734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:38625
I20260620 08:12:32.781602 28922 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:12:32.789251 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.780149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.0.0.1:46118)
0620 08:12:32.780362 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:32.780397 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:32.780474 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:32.780716 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:32.780721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:32.781048 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:32.781091 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.781093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.782525 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.782526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.783153 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.783155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.783832 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.783833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.783905 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.784363 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:32.784367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:32.785899 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:32.787807 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.787824 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.787827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.788247 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.788541 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.788542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.788542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.788683 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.788986 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:32.788991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:32.789031 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 08:12:32.789087 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:12:32.789172 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":84,"threads_started":1}
I20260620 08:12:32.789342 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.780214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.12.132.193:42105)
0620 08:12:32.780359 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:32.780363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:32.780473 (+ 110us) server_negotiation.cc:407] Connection header received
0620 08:12:32.780611 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:32.780614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:32.780630 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:32.780711 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:32.781193 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.782409 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.783233 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.783745 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.783827 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.785986 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:32.785994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:32.785994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:32.786001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:32.787639 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.788337 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.788340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.788340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.788404 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.788750 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.788751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.788751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.788918 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:32.789000 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:32.789154 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 08:12:32.789248 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260620 08:12:32.789705 28848 tablet_service.cc:1514] Processing DeleteTablet for tablet f11e5cb547c54afebc19ca7addcf2bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:12 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:46118
I20260620 08:12:32.790270 28931 tablet_replica.cc:330] stopping tablet replica
I20260620 08:12:32.790372 28931 raft_consensus.cc:2229] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:12:32.790438 28931 raft_consensus.cc:2258] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:12:32.790760 28931 ts_tablet_manager.cc:1905] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:12:32.792760 28931 ts_tablet_manager.cc:1918] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:12:32.792819 28931 log.cc:1196] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal/wals/f11e5cb547c54afebc19ca7addcf2bf1
I20260620 08:12:32.793042 28931 ts_tablet_manager.cc:1939] T f11e5cb547c54afebc19ca7addcf2bf1 P d8a067a0142846daa6640bc8c4d39c70: Deleting consensus metadata
I20260620 08:12:32.793488 28719 catalog_manager.cc:4841] TS d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105): tablet f11e5cb547c54afebc19ca7addcf2bf1 (table test-table [id=f4e41661a621471b9b421a4c5392afc5]) successfully deleted
W20260620 08:12:32.859123 28927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:32.859364 28927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:32.859405 28927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:32.860558 28927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:32.860626 28927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:32.860669 28927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:12:32.862217 28927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:36811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=38105
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:32.862454 28927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:32.862666 28927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:32.868220 28936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:32.868233 28937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:32.868297 28927 server_base.cc:1034] running on GCE node
W20260620 08:12:32.868254 28939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:32.868608 28927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:32.868795 28927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:32.869957 28927 hybrid_clock.cc:648] HybridClock initialized: now 1781943152869942 us; error 29 us; skew 500 ppm
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:32.872308 28927 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:12:32.872776 28927 webserver.cc:460] Webserver started at http://127.12.132.194:38105/ using document root <none> and password file <none>
I20260620 08:12:32.872967 28927 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:32.873018 28927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:32.874310 28927 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:32.875114 28946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:32.875298 28927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:32.875384 28927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "510c411ebc1240b0a568f816d67f615c"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "73c386d206be58e19d0a53a0ed13e6f4"
server_key_iv: "73eb54b5ea2aec7f60486119ac0152b3"
server_key_version: "encryptionkey@0"
I20260620 08:12:32.875710 28927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:32.905457 28927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:32.907655 28927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:32.907796 28927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:32.908007 28927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:32.908322 28927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:32.908356 28927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:32.908378 28927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:32.908392 28927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:32.914387 28927 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36811
I20260620 08:12:32.914482 29059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36811 every 8 connection(s)
I20260620 08:12:32.914804 28927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:12:32.920148 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 28927
I20260620 08:12:32.920461 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:39877
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=39761
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 08:12:32 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943152, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:32.925302 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.915540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:46819)
0620 08:12:32.915736 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:32.915772 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:32.916276 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:32.916534 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:32.916538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:32.917093 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:32.917281 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.917286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.918007 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.918008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.918792 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:32.918794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.919214 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.919215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:32.919345 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.919932 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:32.919941 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:32.921614 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:32.923484 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.923491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.923494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.923814 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.924048 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:32.924049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:32.924049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:32.924193 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:32.924523 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:32.924530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:32.924713 (+ 183us) client_negotiation.cc:769] Sending connection context
0620 08:12:32.924839 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 08:12:32.925020 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260620 08:12:32.925329 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:32.915565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:46819 (local address 127.12.132.254:44069)
0620 08:12:32.915632 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:32.915634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:32.915860 (+ 226us) server_negotiation.cc:407] Connection header received
0620 08:12:32.916382 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:32.916383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:32.916399 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:32.916452 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:32.917384 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.917899 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.918883 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:32.919115 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:32.919202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:32.921754 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:32.921755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:32.921756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:32.921770 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:32.923383 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.923908 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.923909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.923909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.923967 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:32.924311 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:32.924312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:32.924312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:32.924427 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:32.924545 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:32.925088 (+ 543us) server_negotiation.cc:299] Negotiation successful
0620 08:12:32.925238 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:12:32.926326 29060 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:32.926441 29060 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:32.926644 29060 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:32.927264 28734 ts_manager.cc:194] Registered new tserver with Master: 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811)
I20260620 08:12:32.927706 28734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:46819
W20260620 08:12:33.002765 29063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:33.002954 29063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:33.002976 29063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:33.004101 29063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:33.004144 29063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:33.004170 29063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:12:33.005605 29063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:39877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=39761
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:33.005832 29063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:33.006071 29063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:33.011057 29069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:33.011078 29068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:33.011070 29071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:33.011094 29063 server_base.cc:1034] running on GCE node
I20260620 08:12:33.011500 29063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:33.011747 29063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:33.012920 29063 hybrid_clock.cc:648] HybridClock initialized: now 1781943153012907 us; error 48 us; skew 500 ppm
Jun 20 08:12:33 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943153, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:33.015851 29063 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:12:33.016433 29063 webserver.cc:460] Webserver started at http://127.12.132.195:39761/ using document root <none> and password file <none>
I20260620 08:12:33.016671 29063 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:33.016731 29063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:33.018406 29063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:33.019378 29078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:33.019692 29063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:33.019848 29063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "97d697ef8b8d4d488a98e073ea57ae03"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "188e2ed0c4c1332f8485751972a52d5b"
server_key_iv: "5f04cda9aa70578d92213dfd774331b4"
server_key_version: "encryptionkey@0"
I20260620 08:12:33.020371 29063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:33.029273 29063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:33.031212 29063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:33.031379 29063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:33.031678 29063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:33.032018 29063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:33.032075 29063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:33.032111 29063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:33.032138 29063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:33.038331 29063 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:39877
I20260620 08:12:33.038375 29191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:39877 every 8 connection(s)
I20260620 08:12:33.038710 29063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:12:33 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943153, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:12:33.047103 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29063
I20260620 08:12:33.047963 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.039417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60989 (local address 127.12.132.254:44069)
0620 08:12:33.039481 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.039483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.039774 (+ 291us) server_negotiation.cc:407] Connection header received
0620 08:12:33.040184 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.040185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.040195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.040261 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:33.041141 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.041638 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.042624 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.042840 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.042908 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.045097 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:33.045099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:33.045099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:33.045105 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:33.046527 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.046957 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.046957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.046958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.047005 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.047292 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.047292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.047293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.047449 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:33.047517 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.047820 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.047900 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:12:33.048112 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.039410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:60989)
0620 08:12:33.039553 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.039592 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:33.040065 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:33.040317 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:33.040320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:33.040875 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:33.041056 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.041059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.041749 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.041750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.042528 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.042530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.042970 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.042972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.043078 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.043531 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:33.043539 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:33.045010 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:33.046620 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:33.046626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:33.046628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:33.046838 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:33.047072 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:33.047073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:33.047074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:33.047220 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:33.047549 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:33.047556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:33.047709 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 08:12:33.047796 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 08:12:33.047919 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260620 08:12:33.048722 29192 heartbeater.cc:347] Connected to a master server at 127.12.132.254:44069
I20260620 08:12:33.048811 29192 heartbeater.cc:464] Registering TS with master...
I20260620 08:12:33.049006 29192 heartbeater.cc:511] Master 127.12.132.254:44069 requested a full tablet report, sending...
I20260620 08:12:33.049410 28734 ts_manager.cc:194] Registered new tserver with Master: 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877)
I20260620 08:12:33.049795 28734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:60989
I20260620 08:12:33.054965 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.047535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.12.132.254:44069)
0620 08:12:33.047657 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.047658 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.047726 (+ 68us) server_negotiation.cc:407] Connection header received
0620 08:12:33.048068 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.048069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.048083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.048152 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:33.048928 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.049491 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.050286 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.050495 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.050566 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.051633 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:33.051635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:33.051635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:33.051641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:33.053467 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.053946 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.053947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.053948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.054071 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.054313 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.054314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.054315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.054455 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:33.054691 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.054732 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.054832 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260620 08:12:33.055984 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 08:12:33.058935 28663 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.132.254:44069)
I20260620 08:12:33.058996 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.056339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.12.132.254:44069)
0620 08:12:33.056397 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.056398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.056539 (+ 141us) server_negotiation.cc:407] Connection header received
0620 08:12:33.056652 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.056653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.056680 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.056737 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:12:33.056861 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.057311 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.058059 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.058304 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.058385 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.058558 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:12:33.058599 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 08:12:33.058918 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49204: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260620 08:12:33.059034 29196 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49204: token expired
I20260620 08:12:33.067759 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.059173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.12.132.254:44069)
0620 08:12:33.059204 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.059206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.059517 (+ 311us) server_negotiation.cc:407] Connection header received
0620 08:12:33.059568 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.059568 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.059576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.059619 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:33.061260 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.061934 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.062622 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.063080 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.063166 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.064737 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:33.064739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:33.064739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:33.064746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:33.066382 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.066770 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.066772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.066772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.066923 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:33.067260 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:33.067261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:33.067261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:33.067374 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:33.067452 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.067568 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.067674 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260620 08:12:33.068893 28734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:12:33.070468 28734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:12:33.072767 28848 tablet_service.cc:1467] Processing CreateTablet for tablet faf5a1c32c244dbda9bfd1799717acae (DEFAULT_TABLE table=test-table [id=5a16c209a77d438192d9ad50367be904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:33.072964 28848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf5a1c32c244dbda9bfd1799717acae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:33.074288 28814 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Bootstrap starting.
I20260620 08:12:33.074622 28814 tablet_bootstrap.cc:654] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:33.075204 28814 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: No bootstrap required, opened a new log
I20260620 08:12:33.075248 28814 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 08:12:33.075572 28814 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.075630 28814 raft_consensus.cc:374] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:33.075663 28814 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a067a0142846daa6640bc8c4d39c70, State: Initialized, Role: FOLLOWER
I20260620 08:12:33.075704 28814 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.075784 28814 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260620 08:12:33.076040 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.072308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36811 (local address 127.0.0.1:59044)
0620 08:12:33.072356 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.072365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:33.072401 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:33.072990 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:33.072991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:33.072993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:33.073036 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.073037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.074039 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.074040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.075086 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.075089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.075770 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.075771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.075843 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.075863 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 08:12:33.075921 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:12:33.075965 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 08:12:33.076072 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.072600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.12.132.194:36811)
0620 08:12:33.072776 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.072780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.072788 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:12:33.072820 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.072821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.072834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.072982 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:33.073138 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.073957 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.075187 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.075670 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.075757 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.075814 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.075936 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.075972 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260620 08:12:33.076848 28994 tablet_service.cc:1467] Processing CreateTablet for tablet faf5a1c32c244dbda9bfd1799717acae (DEFAULT_TABLE table=test-table [id=5a16c209a77d438192d9ad50367be904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:33.077157 28994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf5a1c32c244dbda9bfd1799717acae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:33.077975 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.072447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.12.132.195:39877)
0620 08:12:33.074377 (+ 1930us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.074381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.074391 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:12:33.074436 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.074438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.074455 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.074539 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:33.074655 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.075733 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.077154 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.077650 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.077707 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.077737 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.077893 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.077917 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1888,"thread_start_us":119,"threads_started":1}
I20260620 08:12:33.077989 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.072352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:39877 (local address 127.0.0.1:34518)
0620 08:12:33.072496 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.072505 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:33.072568 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:33.074547 (+ 1979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:33.074548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:33.074550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:33.074587 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.074588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.075838 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.075839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.077063 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.077066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.077747 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.077748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.077811 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.077815 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:12:33.077886 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 08:12:33.077921 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260620 08:12:33.078646 29126 tablet_service.cc:1467] Processing CreateTablet for tablet faf5a1c32c244dbda9bfd1799717acae (DEFAULT_TABLE table=test-table [id=5a16c209a77d438192d9ad50367be904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:12:33.078933 29126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf5a1c32c244dbda9bfd1799717acae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:33.080013 29202 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Bootstrap starting.
I20260620 08:12:33.080659 29202 tablet_bootstrap.cc:654] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:33.081164 29202 log.cc:824] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:33.081955 29204 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Bootstrap starting.
I20260620 08:12:33.081988 29202 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: No bootstrap required, opened a new log
I20260620 08:12:33.082047 29202 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:12:33.082711 29204 tablet_bootstrap.cc:654] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Neither blocks nor log segments found. Creating new log.
I20260620 08:12:33.083279 29204 log.cc:824] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:33.084002 29204 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: No bootstrap required, opened a new log
I20260620 08:12:33.084074 29204 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:12:33.084249 29202 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.084420 29202 raft_consensus.cc:374] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:33.084452 29202 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510c411ebc1240b0a568f816d67f615c, State: Initialized, Role: FOLLOWER
I20260620 08:12:33.084602 29202 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.084895 29202 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:12:33.084949 29060 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
I20260620 08:12:33.086064 29204 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.086187 29204 raft_consensus.cc:374] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:12:33.086207 29204 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d697ef8b8d4d488a98e073ea57ae03, State: Initialized, Role: FOLLOWER
I20260620 08:12:33.086342 29204 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.086593 29204 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:12:33.086728 29192 heartbeater.cc:503] Master 127.12.132.254:44069 was elected leader, sending a full tablet report...
W20260620 08:12:33.165396 29061 tablet.cc:2369] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:12:33.266320 28923 tablet.cc:2369] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:12:33.289310 29193 tablet.cc:2369] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:12:33.387543 29207 raft_consensus.cc:482] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:33.387676 29207 raft_consensus.cc:504] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.388041 29207 leader_election.cc:288] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811), d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:33.391219 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.388437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:55411 (local address 127.12.132.194:36811)
0620 08:12:33.388519 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.388521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.388715 (+ 194us) server_negotiation.cc:407] Connection header received
0620 08:12:33.388784 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.388785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.388795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.388857 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:33.389082 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.389533 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.390487 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.390856 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.390920 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.390927 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.391101 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.391147 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:12:33.391242 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.388351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36811 (local address 127.12.132.195:55411)
0620 08:12:33.388643 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.388654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:33.388728 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:33.388889 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:33.388890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:33.388893 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:33.389000 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.389002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.389640 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.389640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.390388 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.390390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.390955 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.390956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.391026 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.391031 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:12:33.391087 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:12:33.391148 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":226,"threads_started":1}
I20260620 08:12:33.391304 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.388397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:33337 (local address 127.12.132.193:42105)
0620 08:12:33.388686 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.388689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.388700 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:12:33.388731 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.388732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.388756 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.388867 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:12:33.389017 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.389524 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.390483 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.390912 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.390989 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.391056 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.391178 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.391220 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":193,"threads_started":1}
I20260620 08:12:33.391494 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.388287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.12.132.195:33337)
0620 08:12:33.388370 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.388382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:33.388434 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:33.388849 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:33.388850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:33.388852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:12:33.388928 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.388930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.389612 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.389613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.390385 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:33.390387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.391008 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.391009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:33.391094 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.391097 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:12:33.391153 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 08:12:33.391188 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 08:12:33.391791 29014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "97d697ef8b8d4d488a98e073ea57ae03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "510c411ebc1240b0a568f816d67f615c" is_pre_election: true
I20260620 08:12:33.391989 29014 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d697ef8b8d4d488a98e073ea57ae03 in term 0.
I20260620 08:12:33.391968 28876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "97d697ef8b8d4d488a98e073ea57ae03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a067a0142846daa6640bc8c4d39c70" is_pre_election: true
I20260620 08:12:33.392102 28876 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d697ef8b8d4d488a98e073ea57ae03 in term 0.
I20260620 08:12:33.392234 29079 leader_election.cc:302] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510c411ebc1240b0a568f816d67f615c, 97d697ef8b8d4d488a98e073ea57ae03; no voters:
I20260620 08:12:33.392381 29207 raft_consensus.cc:2781] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:12:33.392465 29207 raft_consensus.cc:482] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:12:33.392494 29207 raft_consensus.cc:3037] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:33.393230 29207 raft_consensus.cc:504] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.393381 29207 leader_election.cc:288] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [CANDIDATE]: Term 1 election: Requested vote from peers 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811), d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:33.393579 29014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "97d697ef8b8d4d488a98e073ea57ae03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "510c411ebc1240b0a568f816d67f615c"
I20260620 08:12:33.393567 28876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "97d697ef8b8d4d488a98e073ea57ae03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8a067a0142846daa6640bc8c4d39c70"
I20260620 08:12:33.393656 29014 raft_consensus.cc:3037] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:33.393656 28876 raft_consensus.cc:3037] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:12:33.394501 29014 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d697ef8b8d4d488a98e073ea57ae03 in term 1.
I20260620 08:12:33.394501 28876 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d697ef8b8d4d488a98e073ea57ae03 in term 1.
I20260620 08:12:33.394694 29080 leader_election.cc:302] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d697ef8b8d4d488a98e073ea57ae03, d8a067a0142846daa6640bc8c4d39c70; no voters:
I20260620 08:12:33.394793 29207 raft_consensus.cc:2781] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:12:33.395004 29207 raft_consensus.cc:686] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 LEADER]: Becoming Leader. State: Replica: 97d697ef8b8d4d488a98e073ea57ae03, State: Running, Role: LEADER
I20260620 08:12:33.395102 29207 consensus_queue.cc:237] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:33.396099 28734 catalog_manager.cc:5495] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "97d697ef8b8d4d488a98e073ea57ae03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } health_report { overall_health: UNKNOWN } } }
I20260620 08:12:33.437160 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.434176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.12.132.195:39877)
0620 08:12:33.434258 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:33.434260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:33.434267 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:12:33.434301 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:33.434302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:33.434321 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:33.434400 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:12:33.434519 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.435192 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.436172 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:33.436355 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:33.436419 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:33.436732 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:12:33.436831 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:12:33.436917 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:33.436996 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 08:12:33.437014 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 08:12:33.439518 29014 raft_consensus.cc:1261] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Refusing update from remote peer 97d697ef8b8d4d488a98e073ea57ae03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:12:33.439675 28876 raft_consensus.cc:1261] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Refusing update from remote peer 97d697ef8b8d4d488a98e073ea57ae03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:12:33.440263 29207 consensus_queue.cc:1035] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:12:33.440557 29210 consensus_queue.cc:1035] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:12:33.442996 29214 mvcc.cc:205] Tried to move back new op lower bound from 7298839156484276224 to 7298839156306976768. Current Snapshot: MvccSnapshot[applied={T|T < 7298839156484276224}]
I20260620 08:12:33.445266 28734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:12:33.445374 28734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:12:33.446108 28734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Sending DeleteTablet for 3 replicas of tablet faf5a1c32c244dbda9bfd1799717acae
I20260620 08:12:33.446426 29126 tablet_service.cc:1514] Processing DeleteTablet for tablet faf5a1c32c244dbda9bfd1799717acae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:33 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:34518
I20260620 08:12:33.446453 28994 tablet_service.cc:1514] Processing DeleteTablet for tablet faf5a1c32c244dbda9bfd1799717acae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:33 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:59044
I20260620 08:12:33.446686 29223 tablet_replica.cc:330] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: stopping tablet replica
I20260620 08:12:33.446709 28848 tablet_service.cc:1514] Processing DeleteTablet for tablet faf5a1c32c244dbda9bfd1799717acae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:12:33 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:46118
I20260620 08:12:33.446828 29223 raft_consensus.cc:2229] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:12:33.446832 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28772
I20260620 08:12:33.446951 29223 raft_consensus.cc:2258] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:12:33.448289 29223 ts_tablet_manager.cc:1905] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:12:33.450371 29223 ts_tablet_manager.cc:1918] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:12:33.450448 29223 log.cc:1196] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/faf5a1c32c244dbda9bfd1799717acae
I20260620 08:12:33.450800 29223 ts_tablet_manager.cc:1939] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Deleting consensus metadata
I20260620 08:12:33.451224 28719 catalog_manager.cc:4841] TS 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877): tablet faf5a1c32c244dbda9bfd1799717acae (table test-table [id=5a16c209a77d438192d9ad50367be904]) successfully deleted
I20260620 08:12:33.452643 29224 tablet_replica.cc:330] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: stopping tablet replica
I20260620 08:12:33.452766 29224 raft_consensus.cc:2229] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:12:33.452878 29224 raft_consensus.cc:2258] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:12:33.453353 29224 ts_tablet_manager.cc:1905] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 08:12:33.453437 28719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:42105 (error 108)
W20260620 08:12:33.453898 29198 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:12:33.453785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.0.0.1:46120)
0620 08:12:33.453827 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.453866 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:42105: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 08:12:33.454154 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28927
W20260620 08:12:33.454030 28719 catalog_manager.cc:4584] TS d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet faf5a1c32c244dbda9bfd1799717acae: Network error: Client connection negotiation failed: client connection to 127.12.132.193:42105: connect: Connection reset by peer (error 104)
I20260620 08:12:33.461295 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:33.461171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36811 (local address 127.0.0.1:59052)
0620 08:12:33.461230 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:33.461262 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:36811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 08:12:33.461295 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29063
I20260620 08:12:33.466063 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 28701
I20260620 08:12:53.471817 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--webserver_interface=127.12.132.254
--webserver_port=42483
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:44069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:53.553638 29227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:53.553850 29227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:53.553872 29227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:53.555016 29227 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 08:12:53.555058 29227 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 08:12:53.555071 29227 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 08:12:53.555083 29227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:12:53.555095 29227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:53.555109 29227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:53.555121 29227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:12:53.555135 29227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:12:53.556600 29227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:44069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:44069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:53.556846 29227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:53.557112 29227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:53.562655 29233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:53.562752 29232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:53.562733 29227 server_base.cc:1034] running on GCE node
W20260620 08:12:53.562891 29235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:53.563103 29227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:53.563412 29227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:53.564590 29227 hybrid_clock.cc:648] HybridClock initialized: now 1781943173564567 us; error 61 us; skew 500 ppm
Jun 20 08:12:53 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943173, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:53.570535 29227 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:12:53.571367 29227 webserver.cc:460] Webserver started at http://127.12.132.254:42483/ using document root <none> and password file <none>
I20260620 08:12:53.571636 29227 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:53.571695 29227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:53.573575 29227 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:12:53.574561 29242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:53.574740 29227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:53.574842 29227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "82fbd8ba7be64a28800fc9fc790ab025"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "6ca2d805cb11a211398268669b984914"
server_key_iv: "552d7fb8463535d178efcf641b95be42"
server_key_version: "encryptionkey@0"
I20260620 08:12:53.575215 29227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:53.590694 29227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:53.593019 29227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:53.593134 29227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:53.597579 29227 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:44069
I20260620 08:12:53.597724 29294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:44069 every 8 connection(s)
I20260620 08:12:53.598048 29227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:12:53.598470 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29227
I20260620 08:12:53.598842 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:42105
--local_ip_for_outbound_sockets=127.12.132.193
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=39237
--webserver_interface=127.12.132.193
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:12:53.600283 29295 sys_catalog.cc:263] Verifying existing consensus state
I20260620 08:12:53.601027 29295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap starting.
I20260620 08:12:53.612605 29295 log.cc:824] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:53.615695 29295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:53.616032 29295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Bootstrap complete.
I20260620 08:12:53.619195 29295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:53.619505 29295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Initialized, Role: FOLLOWER
I20260620 08:12:53.619656 29295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:53.619728 29295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:12:53.619781 29295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:12:53.619837 29295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 2 FOLLOWER]: Advancing to term 3
I20260620 08:12:53.621230 29295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:53.621351 29295 leader_election.cc:302] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fbd8ba7be64a28800fc9fc790ab025; no voters:
I20260620 08:12:53.621513 29295 leader_election.cc:288] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 08:12:53.621599 29299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 08:12:53.621739 29295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:12:53.621810 29299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [term 3 LEADER]: Becoming Leader. State: Replica: 82fbd8ba7be64a28800fc9fc790ab025, State: Running, Role: LEADER
I20260620 08:12:53.621968 29299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } }
I20260620 08:12:53.622326 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:53.622414 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:53.622565 29301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fbd8ba7be64a28800fc9fc790ab025. Latest consensus state: current_term: 3 leader_uuid: "82fbd8ba7be64a28800fc9fc790ab025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fbd8ba7be64a28800fc9fc790ab025" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 44069 } } }
I20260620 08:12:53.622622 29301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025 [sys.catalog]: This master's current role is: LEADER
I20260620 08:12:53.622607 29308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:12:53.623384 29308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:12:53.623550 29308 catalog_manager.cc:1259] Loaded cluster ID: 33d058863b954f46bd8299fe734d8abe
I20260620 08:12:53.623756 29308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:12:53.624729 29308 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:12:53.624909 29308 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Loaded TSK: 0
I20260620 08:12:53.624963 29308 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fbd8ba7be64a28800fc9fc790ab025: Loaded TSK: 1
F20260620 08:12:53.625159 29308 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2a3ef809ad google::LogMessage::Fail() at ??:0
@ 0x7f2a3ef8388a google::LogMessage::SendToLog() at ??:0
@ 0x7f2a3ef804de google::LogMessage::Flush() at ??:0
@ 0x7f2a3ef81a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2a416ed104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2a416ead41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2a46eda1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2a46eda48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2a46eda594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2a46ea0c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2a46eaaad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2a40e4e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2a40e43954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2a41ca76db start_thread at ??:0
@ 0x7f2a3da7e61f clone at ??:0
W20260620 08:12:53.709684 29297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:53.709892 29297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:53.709916 29297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:53.711071 29297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:53.711114 29297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:53.711140 29297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:12:53.712622 29297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:42105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=39237
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:53.712853 29297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:53.713089 29297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:53.718185 29339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:53.718204 29337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:53.718206 29336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:53.718425 29297 server_base.cc:1034] running on GCE node
I20260620 08:12:53.718719 29297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:53.718978 29297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:53.720165 29297 hybrid_clock.cc:648] HybridClock initialized: now 1781943173720142 us; error 39 us; skew 500 ppm
Jun 20 08:12:53 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943173, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:53.722687 29297 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:12:53.723135 29297 webserver.cc:460] Webserver started at http://127.12.132.193:39237/ using document root <none> and password file <none>
I20260620 08:12:53.723376 29297 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:53.723448 29297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:53.725173 29297 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:12:53.726066 29346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:53.726379 29297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:12:53.726495 29297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "d8a067a0142846daa6640bc8c4d39c70"
format_stamp: "Formatted at 2026-06-20 08:12:11 on dist-test-slave-z0sp"
server_key: "c357693767c72f21d8e56434dc1c2168"
server_key_iv: "ea51a54dd983af2ac5df393f78a887e0"
server_key_version: "encryptionkey@0"
I20260620 08:12:53.726848 29297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:53.743232 29297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:53.745980 29297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:53.746178 29297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:53.746451 29297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:53.746925 29353 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 08:12:53.748920 29297 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 08:12:53.748992 29297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 08:12:53.749042 29297 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 08:12:53.749676 29297 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 08:12:53.749732 29297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 08:12:53.749812 29353 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Bootstrap starting.
I20260620 08:12:53.757033 29297 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:42105
I20260620 08:12:53.757162 29460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:42105 every 8 connection(s)
I20260620 08:12:53.757447 29297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:12:53.761650 29353 log.cc:824] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:53.763681 29353 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:53.763976 29353 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Bootstrap complete.
I20260620 08:12:53.764984 29353 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260620 08:12:53.765971 29353 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:53.766186 29353 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8a067a0142846daa6640bc8c4d39c70, State: Initialized, Role: FOLLOWER
I20260620 08:12:53.766319 29353 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:53.766626 29353 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:12:53.766657 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29297
I20260620 08:12:53.766969 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:36811
--local_ip_for_outbound_sockets=127.12.132.194
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=38105
--webserver_interface=127.12.132.194
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:53.854905 29463 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:12:53.758143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:48029)
0620 08:12:53.758324 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.758360 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:53.758801 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:53.854533 (+ 95732us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
W20260620 08:12:53.855278 29461 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:44069 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:44069: Client connection negotiation failed: client connection to 127.12.132.254:44069: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:12:53.855716 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:53.855579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:47697)
0620 08:12:53.855642 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.855688 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:12:53.856159 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:53.856056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:45449)
0620 08:12:53.856117 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.856144 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 08:12:53.856290 29461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 08:12:53.870318 29466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:53.870508 29466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:53.870532 29466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:53.871670 29466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:53.871716 29466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:53.871742 29466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:12:53.873353 29466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:36811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=38105
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:53.873584 29466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:53.873812 29466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:53.879030 29472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:53.879060 29471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:53.879060 29474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:53.879119 29466 server_base.cc:1034] running on GCE node
I20260620 08:12:53.879475 29466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:53.879740 29466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:53.880921 29466 hybrid_clock.cc:648] HybridClock initialized: now 1781943173880909 us; error 31 us; skew 500 ppm
Jun 20 08:12:53 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943173, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:53.883513 29466 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:12:53.883949 29466 webserver.cc:460] Webserver started at http://127.12.132.194:38105/ using document root <none> and password file <none>
I20260620 08:12:53.884183 29466 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:53.884251 29466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:53.885651 29466 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:53.886453 29481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:53.886742 29466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:53.886858 29466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "510c411ebc1240b0a568f816d67f615c"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "73c386d206be58e19d0a53a0ed13e6f4"
server_key_iv: "73eb54b5ea2aec7f60486119ac0152b3"
server_key_version: "encryptionkey@0"
I20260620 08:12:53.887197 29466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:53.909189 29466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:53.911535 29466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:53.911700 29466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:53.911952 29466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:53.912385 29488 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 08:12:53.914104 29466 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 08:12:53.914256 29466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 08:12:53.914309 29466 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 08:12:53.915242 29466 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 08:12:53.915307 29466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 08:12:53.915381 29488 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Bootstrap starting.
I20260620 08:12:53.916739 29488 log.cc:824] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:53.919734 29488 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:53.920055 29488 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Bootstrap complete.
I20260620 08:12:53.920204 29488 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 08:12:53.921597 29488 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:53.921861 29488 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 510c411ebc1240b0a568f816d67f615c, State: Initialized, Role: FOLLOWER
I20260620 08:12:53.922032 29488 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:53.922341 29488 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260620 08:12:53.923332 29466 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:36811
I20260620 08:12:53.923489 29597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:36811 every 8 connection(s)
I20260620 08:12:53.923734 29466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:12:53.924758 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:53.924358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:50619)
0620 08:12:53.924554 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.924634 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
W20260620 08:12:53.924993 29598 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:44069 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:44069: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
I20260620 08:12:53.925331 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:53.925200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:54201)
0620 08:12:53.925280 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.925309 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 08:12:53.925709 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:53.925617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:38175)
0620 08:12:53.925661 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:53.925684 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 08:12:53.925853 29598 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 08:12:53.933931 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29466
I20260620 08:12:53.934263 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:39877
--local_ip_for_outbound_sockets=127.12.132.195
--tserver_master_addrs=127.12.132.254:44069
--webserver_port=39761
--webserver_interface=127.12.132.195
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:12:54.020449 29601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:12:54.020740 29601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:12:54.020766 29601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:12:54.021919 29601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:12:54.021966 29601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:12:54.021993 29601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:12:54.023427 29601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:39877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=39761
--tserver_master_addrs=127.12.132.254:44069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:12:54.023842 29601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:12:54.024282 29601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:12:54.029507 29609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:54.029511 29606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:12:54.029507 29607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:12:54.029768 29601 server_base.cc:1034] running on GCE node
I20260620 08:12:54.029973 29601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:12:54.030203 29601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:12:54.031380 29601 hybrid_clock.cc:648] HybridClock initialized: now 1781943174031375 us; error 36 us; skew 500 ppm
Jun 20 08:12:54 dist-test-slave-z0sp krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943174, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:12:54.033963 29601 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:12:54.034447 29601 webserver.cc:460] Webserver started at http://127.12.132.195:39761/ using document root <none> and password file <none>
I20260620 08:12:54.034670 29601 fs_manager.cc:359] Metadata directory not provided
I20260620 08:12:54.034739 29601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:12:54.036675 29601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:12:54.037667 29616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:54.037940 29601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:12:54.038048 29601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "97d697ef8b8d4d488a98e073ea57ae03"
format_stamp: "Formatted at 2026-06-20 08:12:12 on dist-test-slave-z0sp"
server_key: "188e2ed0c4c1332f8485751972a52d5b"
server_key_iv: "5f04cda9aa70578d92213dfd774331b4"
server_key_version: "encryptionkey@0"
I20260620 08:12:54.038403 29601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:12:54.063598 29601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:12:54.065968 29601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:12:54.066133 29601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:12:54.066380 29601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:12:54.066705 29601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:12:54.066763 29601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:54.066802 29601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:12:54.066831 29601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:12:54.072779 29601 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:39877
I20260620 08:12:54.072836 29729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:39877 every 8 connection(s)
I20260620 08:12:54.073144 29601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:12:54.074501 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:54.073787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:57947)
0620 08:12:54.074083 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:54.074285 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
W20260620 08:12:54.075076 29730 heartbeater.cc:644] Failed to heartbeat to 127.12.132.254:44069 (0 consecutive failures): Network error: Failed to ping master at 127.12.132.254:44069: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
I20260620 08:12:54.075453 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:54.075345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:43299)
0620 08:12:54.075401 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:54.075430 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 08:12:54.075879 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:54.075749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:51819)
0620 08:12:54.075823 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:54.075856 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 08:12:54.076077 29730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 08:12:54.080862 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29601
I20260620 08:12:54.857260 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:54.856739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:36005)
0620 08:12:54.856998 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:54.857081 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":80,"threads_started":1}
I20260620 08:12:54.926432 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:54.926210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:49957)
0620 08:12:54.926353 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:54.926388 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":54,"threads_started":1}
I20260620 08:12:55.076958 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.076669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:37391)
0620 08:12:55.076857 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.076910 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
I20260620 08:12:55.310199 29737 raft_consensus.cc:482] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:12:55.310375 29737 raft_consensus.cc:504] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:55.310761 29737 leader_election.cc:288] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811), 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877)
Jun 20 08:12:55 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943173, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.194@KRBTEST.COM
Jun 20 08:12:55 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943173, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.195@KRBTEST.COM
I20260620 08:12:55.321861 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.311066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:39535 (local address 127.12.132.195:39877)
0620 08:12:55.311291 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:55.311304 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:55.311350 (+ 46us) server_negotiation.cc:407] Connection header received
0620 08:12:55.311916 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:55.311919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:55.311952 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:55.312042 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:55.312628 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.314011 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315000 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.315399 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315524 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:55.318276 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:55.318280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:55.318280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:55.318290 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:55.320080 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:55.320676 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:55.320679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:55.320679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:55.320766 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:55.321005 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:55.321006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:55.321006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:55.321239 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:55.321331 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:55.321616 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 08:12:55.321740 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":143,"threads_started":1}
I20260620 08:12:55.321851 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.310946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:39877 (local address 127.12.132.193:39535)
0620 08:12:55.311005 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.311016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:55.311070 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:55.312075 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:55.312076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:55.312241 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:55.312456 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:55.312464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.314147 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.314148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:55.314902 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:55.314905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.315498 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:55.315613 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:55.316310 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:55.316319 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:55.318165 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:55.320185 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:55.320193 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:55.320195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:55.320600 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:55.320827 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:55.320828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:55.320828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:55.320924 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:55.321312 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:55.321318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:55.321454 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 08:12:55.321553 (+ 99us) client_negotiation.cc:240] Negotiation successful
0620 08:12:55.321740 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":54}
I20260620 08:12:55.321947 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.311053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:36811 (local address 127.12.132.193:52147)
0620 08:12:55.311170 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.311180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:55.311231 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:55.311767 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:55.311770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:55.312158 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:55.312456 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:55.312464 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.314131 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.314132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:55.314905 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:55.314907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.315497 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:55.315611 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:55.316265 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:55.316273 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:55.317838 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:55.320233 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:55.320234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:55.320234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:55.320600 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:55.321005 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:55.321007 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:55.321007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:55.321071 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:55.321454 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:55.321455 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:55.321519 (+ 64us) client_negotiation.cc:769] Sending connection context
0620 08:12:55.321589 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 08:12:55.321823 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":139,"thread_start_us":64,"threads_started":1}
I20260620 08:12:55.321941 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.311221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52147 (local address 127.12.132.194:36811)
0620 08:12:55.311342 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:55.311349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:55.311369 (+ 20us) server_negotiation.cc:407] Connection header received
0620 08:12:55.311481 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:55.311483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:55.311531 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:55.311668 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:55.312621 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.314029 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315000 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:55.315399 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:55.315532 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:55.317967 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:55.317971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:55.317971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:55.317984 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:55.320168 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:55.320854 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:55.320856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:55.320856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:55.320930 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:55.321146 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:55.321147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:55.321147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:55.321368 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:55.321460 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:55.321666 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 08:12:55.321810 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":54,"threads_started":1}
I20260620 08:12:55.322412 29548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "d8a067a0142846daa6640bc8c4d39c70" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "510c411ebc1240b0a568f816d67f615c" is_pre_election: true
I20260620 08:12:55.322569 29548 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8a067a0142846daa6640bc8c4d39c70 in term 1.
I20260620 08:12:55.322959 29347 leader_election.cc:302] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 510c411ebc1240b0a568f816d67f615c, d8a067a0142846daa6640bc8c4d39c70; no voters:
I20260620 08:12:55.323167 29737 raft_consensus.cc:2781] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 08:12:55.323251 29737 raft_consensus.cc:482] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:12:55.323277 29737 raft_consensus.cc:3037] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 1 FOLLOWER]: Advancing to term 2
W20260620 08:12:55.323266 29348 leader_election.cc:341] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877): Not found: Tablet not found: faf5a1c32c244dbda9bfd1799717acae
I20260620 08:12:55.324591 29737 raft_consensus.cc:504] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:55.324775 29737 leader_election.cc:288] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 election: Requested vote from peers 510c411ebc1240b0a568f816d67f615c (127.12.132.194:36811), 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877)
I20260620 08:12:55.324944 29548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf5a1c32c244dbda9bfd1799717acae" candidate_uuid: "d8a067a0142846daa6640bc8c4d39c70" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "510c411ebc1240b0a568f816d67f615c"
W20260620 08:12:55.325047 29348 leader_election.cc:341] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877): Not found: Tablet not found: faf5a1c32c244dbda9bfd1799717acae
I20260620 08:12:55.325038 29548 raft_consensus.cc:3037] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 1 FOLLOWER]: Advancing to term 2
I20260620 08:12:55.326195 29548 raft_consensus.cc:2454] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8a067a0142846daa6640bc8c4d39c70 in term 2.
I20260620 08:12:55.326390 29347 leader_election.cc:302] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 510c411ebc1240b0a568f816d67f615c, d8a067a0142846daa6640bc8c4d39c70; no voters: 97d697ef8b8d4d488a98e073ea57ae03
I20260620 08:12:55.326505 29737 raft_consensus.cc:2781] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 08:12:55.326647 29737 raft_consensus.cc:686] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [term 2 LEADER]: Becoming Leader. State: Replica: d8a067a0142846daa6640bc8c4d39c70, State: Running, Role: LEADER
I20260620 08:12:55.326756 29737 consensus_queue.cc:237] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:55.327104 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.326979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:41623)
0620 08:12:55.327029 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.327070 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 08:12:55.733405 29348 consensus_peers.cc:487] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 -> Peer 97d697ef8b8d4d488a98e073ea57ae03 (127.12.132.195:39877): Couldn't send request to peer 97d697ef8b8d4d488a98e073ea57ae03. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: faf5a1c32c244dbda9bfd1799717acae. This is attempt 1: this message will repeat every 5th retry.
I20260620 08:12:55.771335 29548 raft_consensus.cc:1261] T faf5a1c32c244dbda9bfd1799717acae P 510c411ebc1240b0a568f816d67f615c [term 2 FOLLOWER]: Refusing update from remote peer d8a067a0142846daa6640bc8c4d39c70: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260620 08:12:55.771785 29737 consensus_queue.cc:1035] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260620 08:12:55.772948 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.772742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:32905)
0620 08:12:55.772847 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.772900 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":41,"threads_started":1}
I20260620 08:12:55.773784 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.773673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:51939)
0620 08:12:55.773721 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.773751 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 08:12:55.773893 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:55.773785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:55139)
0620 08:12:55.773840 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:55.773869 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:12:56.077840 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.077524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:48699)
0620 08:12:56.077700 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.077776 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":60,"threads_started":1}
I20260620 08:12:56.262697 29749 ts_tablet_manager.cc:927] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Initiating tablet copy from peer d8a067a0142846daa6640bc8c4d39c70 (127.12.132.193:42105)
I20260620 08:12:56.263059 29749 tablet_copy_client.cc:316] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: tablet copy: Beginning tablet copy session from remote peer at address 127.12.132.193:42105
Jun 20 08:12:56 dist-test-slave-z0sp krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943174, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.193@KRBTEST.COM
I20260620 08:12:56.271581 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.263434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:42105 (local address 127.12.132.195:35465)
0620 08:12:56.263626 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.263650 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 08:12:56.263757 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:12:56.264104 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:12:56.264114 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:12:56.264437 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:12:56.264588 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:56.264591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:56.265456 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:56.265456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:56.266044 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:12:56.266046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:12:56.266521 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:12:56.266522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:12:56.266573 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:56.267078 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:12:56.267091 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:12:56.268479 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:12:56.270378 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:56.270384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:56.270386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:56.270580 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:56.270830 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:12:56.270831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:12:56.270831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:12:56.270886 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:12:56.271163 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:12:56.271174 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:12:56.271249 (+ 75us) client_negotiation.cc:769] Sending connection context
0620 08:12:56.271322 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 08:12:56.271474 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260620 08:12:56.271739 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.263772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:35465 (local address 127.12.132.193:42105)
0620 08:12:56.263900 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0620 08:12:56.263905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:12:56.263917 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:12:56.263943 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:12:56.263945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:12:56.263957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:12:56.264039 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:12:56.264677 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:56.265359 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:56.266141 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:12:56.266444 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:12:56.266520 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:12:56.268656 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:12:56.268662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:12:56.268663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:12:56.268669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:12:56.270257 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:56.270686 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:56.270689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:56.270690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:56.270748 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:12:56.270942 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:12:56.270942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:12:56.270943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:12:56.271083 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:12:56.271145 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:12:56.271387 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 08:12:56.271468 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":53,"threads_started":1}
I20260620 08:12:56.272282 29435 tablet_copy_service.cc:140] P d8a067a0142846daa6640bc8c4d39c70: Received BeginTabletCopySession request for tablet faf5a1c32c244dbda9bfd1799717acae from peer 97d697ef8b8d4d488a98e073ea57ae03 ({username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:35465)
I20260620 08:12:56.272373 29435 tablet_copy_service.cc:161] P d8a067a0142846daa6640bc8c4d39c70: Beginning new tablet copy session on tablet faf5a1c32c244dbda9bfd1799717acae from peer 97d697ef8b8d4d488a98e073ea57ae03 at {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:35465: session id = 97d697ef8b8d4d488a98e073ea57ae03-faf5a1c32c244dbda9bfd1799717acae
I20260620 08:12:56.272904 29435 tablet_copy_source_session.cc:205] T faf5a1c32c244dbda9bfd1799717acae P d8a067a0142846daa6640bc8c4d39c70: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 08:12:56.274096 29749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf5a1c32c244dbda9bfd1799717acae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:12:56.276768 29749 tablet_copy_client.cc:791] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: tablet copy: Starting download of 0 data blocks...
I20260620 08:12:56.276932 29749 tablet_copy_client.cc:655] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: tablet copy: Starting download of 1 WAL segments...
I20260620 08:12:56.278070 29749 tablet_copy_client.cc:531] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 08:12:56.279170 29749 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Bootstrap starting.
I20260620 08:12:56.280587 29749 log.cc:824] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Log is configured to *not* fsync() on all Append() calls
I20260620 08:12:56.283169 29749 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 08:12:56.283406 29749 tablet_bootstrap.cc:492] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Bootstrap complete.
I20260620 08:12:56.283556 29749 ts_tablet_manager.cc:1397] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260620 08:12:56.284619 29749 raft_consensus.cc:348] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:56.284811 29749 raft_consensus.cc:729] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d697ef8b8d4d488a98e073ea57ae03, State: Initialized, Role: FOLLOWER
I20260620 08:12:56.284988 29749 consensus_queue.cc:260] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "510c411ebc1240b0a568f816d67f615c" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 36811 } } peers { permanent_uuid: "97d697ef8b8d4d488a98e073ea57ae03" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 39877 } } peers { permanent_uuid: "d8a067a0142846daa6640bc8c4d39c70" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 42105 } }
I20260620 08:12:56.285251 29749 ts_tablet_manager.cc:1428] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260620 08:12:56.285609 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.285491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:36755)
0620 08:12:56.285552 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.285577 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 08:12:56.285589 29435 tablet_copy_service.cc:342] P d8a067a0142846daa6640bc8c4d39c70: Request end of tablet copy session 97d697ef8b8d4d488a98e073ea57ae03-faf5a1c32c244dbda9bfd1799717acae received from {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:35465
I20260620 08:12:56.285812 29435 tablet_copy_service.cc:433] P d8a067a0142846daa6640bc8c4d39c70: ending tablet copy session 97d697ef8b8d4d488a98e073ea57ae03-faf5a1c32c244dbda9bfd1799717acae on tablet faf5a1c32c244dbda9bfd1799717acae with peer 97d697ef8b8d4d488a98e073ea57ae03
I20260620 08:12:56.286005 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.285933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:56171)
0620 08:12:56.285962 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.285985 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260620 08:12:56.774566 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.774335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:39483)
0620 08:12:56.774487 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.774520 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260620 08:12:56.774632 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.774387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:55483)
0620 08:12:56.774520 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.774573 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260620 08:12:56.804478 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.804349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:53593)
0620 08:12:56.804411 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.804441 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 08:12:56.804544 29684 raft_consensus.cc:1203] T faf5a1c32c244dbda9bfd1799717acae P 97d697ef8b8d4d488a98e073ea57ae03 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260620 08:12:56.805331 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:56.805142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:40487)
0620 08:12:56.805267 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:56.805302 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260620 08:12:57.775374 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:57.775151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:52493)
0620 08:12:57.775283 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:57.775326 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260620 08:12:57.805408 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:57.805173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:47849)
0620 08:12:57.805302 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:57.805354 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260620 08:12:57.805931 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:57.805719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:54481)
0620 08:12:57.805862 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:57.805897 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":48,"threads_started":1}
I20260620 08:12:58.776118 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:58.775891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:44689)
0620 08:12:58.776035 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:58.776072 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260620 08:12:58.806214 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:58.805996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:56157)
0620 08:12:58.806120 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:58.806158 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260620 08:12:58.806574 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:58.806399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:56185)
0620 08:12:58.806514 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:58.806540 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":49,"threads_started":1}
I20260620 08:12:58.806648 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:58.806527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:42173)
0620 08:12:58.806576 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:58.806607 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 08:12:59.777173 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:59.776724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:35003)
0620 08:12:59.776935 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:59.777126 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":65,"threads_started":1}
I20260620 08:12:59.807435 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:59.807107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:49153)
0620 08:12:59.807281 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:59.807345 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":58,"threads_started":1}
I20260620 08:12:59.807595 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:12:59.807268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:51285)
0620 08:12:59.807456 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 08:12:59.807502 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260620 08:13:00.778179 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:00.777849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:44459)
0620 08:13:00.778076 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:00.778129 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":151,"threads_started":1}
I20260620 08:13:00.808172 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:00.807953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:51451)
0620 08:13:00.808089 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:00.808126 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260620 08:13:00.808472 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:00.808258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:51433)
0620 08:13:00.808397 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:00.808434 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260620 08:13:01.779292 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:01.778891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:59291)
0620 08:13:01.779194 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:01.779245 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":194,"threads_started":1}
I20260620 08:13:01.809077 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:01.808843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:59901)
0620 08:13:01.808991 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:01.809031 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260620 08:13:01.809118 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:01.808895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:33477)
0620 08:13:01.809035 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:01.809072 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260620 08:13:02.780210 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:02.779951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:49525)
0620 08:13:02.780123 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:02.780163 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260620 08:13:02.809903 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:02.809679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:45373)
0620 08:13:02.809811 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:02.809856 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260620 08:13:02.810271 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:02.809871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:45205)
0620 08:13:02.810048 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:02.810100 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260620 08:13:03.781333 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:03.780941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:45399)
0620 08:13:03.781208 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:03.781266 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260620 08:13:03.811050 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:03.810702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:60645)
0620 08:13:03.810923 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:03.810973 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
I20260620 08:13:03.811144 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:03.810848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:41277)
0620 08:13:03.811033 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:03.811096 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
I20260620 08:13:04.782310 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:04.782047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:34655)
0620 08:13:04.782212 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:04.782251 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260620 08:13:04.811985 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:04.811742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:55237)
0620 08:13:04.811907 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:04.811941 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260620 08:13:04.812050 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:04.811847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:34789)
0620 08:13:04.811973 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:04.812010 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260620 08:13:05.783254 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:05.782939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:33521)
0620 08:13:05.783138 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:05.783190 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260620 08:13:05.812980 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:05.812541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:33793)
0620 08:13:05.812733 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:05.812797 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260620 08:13:05.813083 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:05.812728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:54187)
0620 08:13:05.812928 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:05.812981 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260620 08:13:06.784097 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:06.783817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:54249)
0620 08:13:06.784006 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:06.784048 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":81,"threads_started":1}
I20260620 08:13:06.813771 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:06.813547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:44575)
0620 08:13:06.813691 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:06.813724 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260620 08:13:06.814013 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:06.813678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:34735)
0620 08:13:06.813839 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:06.813923 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260620 08:13:07.785100 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:07.784822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:34197)
0620 08:13:07.785003 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:07.785050 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260620 08:13:07.814615 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:07.814323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:58859)
0620 08:13:07.814507 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:07.814548 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260620 08:13:07.814842 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:07.814543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:50291)
0620 08:13:07.814750 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:07.814795 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260620 08:13:08.785966 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:08.785708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.194:59331)
0620 08:13:08.785882 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:08.785919 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":77,"threads_started":1}
I20260620 08:13:08.815339 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:08.815114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.195:60697)
0620 08:13:08.815248 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:08.815296 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260620 08:13:08.815721 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:08.815494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:44069 (local address 127.12.132.193:60959)
0620 08:13:08.815614 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:08.815643 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.254:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":50,"threads_started":1}
I20260620 08:13:09.082405 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29297
I20260620 08:13:09.088617 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29466
I20260620 08:13:09.093779 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29601
2026-06-20T08:13:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58700 ms)
[----------] 30 tests from SecurityITest (239436 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:09 dist-test-slave-z0sp krb5kdc[29799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:09 dist-test-slave-z0sp krb5kdc[29799](info): set up 2 sockets
Jun 20 08:13:09 dist-test-slave-z0sp krb5kdc[29799](info): commencing operation
krb5kdc: starting...
W20260620 08:13:10.157157 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:10.288136 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:42877
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:42877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:10.367961 29812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:10.368155 29812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:10.368176 29812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:10.369251 29812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:10.369290 29812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:10.369306 29812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:10.369318 29812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:10.369330 29812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:10.370728 29812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:42877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:42877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:10.370911 29812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:10.371074 29812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:10.375962 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.376016 29812 server_base.cc:1034] running on GCE node
W20260620 08:13:10.376181 29820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:10.376256 29817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.376627 29812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:10.376957 29812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:10.378139 29812 hybrid_clock.cc:648] HybridClock initialized: now 1781943190378107 us; error 61 us; skew 500 ppm
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:10.380708 29812 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:10.381101 29812 webserver.cc:460] Webserver started at http://127.12.132.254:35183/ using document root <none> and password file <none>
I20260620 08:13:10.381350 29812 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:10.381397 29812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:10.381542 29812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:10.382691 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "87dd56e76cbf47a2a1820ab20da574ae"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "d76c5fbd2fa30b6702f8d3e42cdb7ee9"
server_key_iv: "b5064fc8b06ddb252b2fe208815fe50b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.383052 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "87dd56e76cbf47a2a1820ab20da574ae"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "d76c5fbd2fa30b6702f8d3e42cdb7ee9"
server_key_iv: "b5064fc8b06ddb252b2fe208815fe50b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.384379 29812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:10.385185 29827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.385380 29812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:10.385473 29812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "87dd56e76cbf47a2a1820ab20da574ae"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "d76c5fbd2fa30b6702f8d3e42cdb7ee9"
server_key_iv: "b5064fc8b06ddb252b2fe208815fe50b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.385527 29812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:10.397749 29812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:10.399995 29812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:10.400102 29812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:10.404325 29812 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:42877
I20260620 08:13:10.404414 29879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:42877 every 8 connection(s)
I20260620 08:13:10.404771 29812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:10.405604 29880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:10.407811 29880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: Bootstrap starting.
I20260620 08:13:10.408391 29880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:10.408751 29880 log.cc:824] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:10.409768 29880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: No bootstrap required, opened a new log
I20260620 08:13:10.411861 29880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } }
I20260620 08:13:10.412155 29880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:10.412241 29880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87dd56e76cbf47a2a1820ab20da574ae, State: Initialized, Role: FOLLOWER
I20260620 08:13:10.412411 29880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } }
I20260620 08:13:10.412483 29880 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:10.412569 29880 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:10.412626 29880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:10.412971 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29812
I20260620 08:13:10.413055 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:10.413272 12819 external_mini_cluster.cc:1428] Setting key fd4675970589214d28d2f9ce06f154c3
I20260620 08:13:10.413544 29880 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } }
I20260620 08:13:10.413673 29880 leader_election.cc:302] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87dd56e76cbf47a2a1820ab20da574ae; no voters:
I20260620 08:13:10.414139 29880 leader_election.cc:288] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:10.414160 29885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:10.414284 29885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [term 1 LEADER]: Becoming Leader. State: Replica: 87dd56e76cbf47a2a1820ab20da574ae, State: Running, Role: LEADER
I20260620 08:13:10.414415 29880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:10.414449 29885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } }
I20260620 08:13:10.415055 29886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87dd56e76cbf47a2a1820ab20da574ae. Latest consensus state: current_term: 1 leader_uuid: "87dd56e76cbf47a2a1820ab20da574ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } } }
I20260620 08:13:10.415148 29886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:10.415546 29887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87dd56e76cbf47a2a1820ab20da574ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87dd56e76cbf47a2a1820ab20da574ae" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 42877 } } }
I20260620 08:13:10.415674 29887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:10.416178 29901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:10.416249 29901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:10.416297 29894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:10.416486 29894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:10.421829 29894 catalog_manager.cc:1347] Generated new cluster ID: 040d078d078e4c6cb3caf7d8671579ce
I20260620 08:13:10.421885 29894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:10.425473 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.413864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.12.132.254:42877)
0620 08:13:10.414089 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.414097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.414154 (+ 57us) server_negotiation.cc:407] Connection header received
0620 08:13:10.414314 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.414322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.414382 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.415339 (+ 957us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:10.415489 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.416885 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.417971 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.418391 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.418478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.421137 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:10.421143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:10.421143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:10.421162 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:10.423641 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.424122 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.424127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.424127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.424299 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.424532 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.424533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.424533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.424850 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:10.425025 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.425099 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.425270 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
I20260620 08:13:10.443944 29894 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:10.444456 29894 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:10.450569 29894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: Generated new TSK 0
I20260620 08:13:10.450853 29894 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:13:10.485517 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:10.570346 29906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:10.570572 29906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:10.570612 29906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:10.571751 29906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:10.571827 29906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:10.571873 29906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:10.573465 29906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:10.573722 29906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:10.573947 29906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:10.579360 29912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:10.579488 29914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.579564 29906 server_base.cc:1034] running on GCE node
W20260620 08:13:10.579777 29911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.580054 29906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:10.580363 29906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:10.581552 29906 hybrid_clock.cc:648] HybridClock initialized: now 1781943190581555 us; error 68 us; skew 500 ppm
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:10.584483 29906 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:13:10.585031 29906 webserver.cc:460] Webserver started at http://127.12.132.193:41927/ using document root <none> and password file <none>
I20260620 08:13:10.585270 29906 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:10.585342 29906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:10.585798 29906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:10.587181 29906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "4cb01bdd45f44b1c81b1a71e758d3c80"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "acf948bd738b9ee4492d9cdf5af3c962"
server_key_iv: "2a5d30d876104a724cc539086dfb3d5b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.587672 29906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "4cb01bdd45f44b1c81b1a71e758d3c80"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "acf948bd738b9ee4492d9cdf5af3c962"
server_key_iv: "2a5d30d876104a724cc539086dfb3d5b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.589571 29906 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260620 08:13:10.590648 29921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.590893 29906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:10.591019 29906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "4cb01bdd45f44b1c81b1a71e758d3c80"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "acf948bd738b9ee4492d9cdf5af3c962"
server_key_iv: "2a5d30d876104a724cc539086dfb3d5b"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.591120 29906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:10.601794 29906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:10.604075 29906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:10.604246 29906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:10.604560 29906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:10.604885 29906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:10.604943 29906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.604984 29906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:10.605011 29906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.611150 29906 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:33151
I20260620 08:13:10.611191 30034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:33151 every 8 connection(s)
I20260620 08:13:10.611550 29906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:10.621062 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 29906
I20260620 08:13:10.621207 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:10.621456 12819 external_mini_cluster.cc:1428] Setting key 86d3629759a1b4ce6307b6f570d9e348
I20260620 08:13:10.623425 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.612310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:51199 (local address 127.12.132.254:42877)
0620 08:13:10.612393 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.612395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.612585 (+ 190us) server_negotiation.cc:407] Connection header received
0620 08:13:10.613116 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.613117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.613130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.613188 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:10.614092 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.614685 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.616195 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.616397 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.616456 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.619657 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:10.619659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:10.619659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:10.619666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:10.621622 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.622281 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.622283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.622283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.622357 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.622692 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.622693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.622693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.622841 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:10.622928 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.623188 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.623290 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 08:13:10.623438 30037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.612265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42877 (local address 127.12.132.193:51199)
0620 08:13:10.612440 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.612482 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.613026 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.613243 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.613246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.613795 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:10.613983 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.613987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.614923 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.614926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.616114 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.616119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.616641 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.616643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.616836 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.617709 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:10.617720 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:10.619511 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:10.621768 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.621778 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.621780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.622143 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.622458 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.622459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.622459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.622610 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.622929 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:10.622935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:10.623052 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.623138 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.623272 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260620 08:13:10.624577 30035 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42877
I20260620 08:13:10.624708 30035 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:10.624907 30035 heartbeater.cc:511] Master 127.12.132.254:42877 requested a full tablet report, sending...
I20260620 08:13:10.625499 29844 ts_manager.cc:194] Registered new tserver with Master: 4cb01bdd45f44b1c81b1a71e758d3c80 (127.12.132.193:33151)
I20260620 08:13:10.626441 29844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:51199
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:13:10.653095 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:10.738189 30040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:10.738456 30040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:10.738504 30040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:10.739972 30040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:10.740044 30040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:10.740092 30040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:10.741703 30040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:10.741972 30040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:10.742205 30040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:10.747450 30045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:10.747514 30048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.747630 30040 server_base.cc:1034] running on GCE node
W20260620 08:13:10.747547 30046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.748159 30040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:10.748660 30040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:10.749874 30040 hybrid_clock.cc:648] HybridClock initialized: now 1781943190749867 us; error 61 us; skew 500 ppm
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:10.752616 30040 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:13:10.753079 30040 webserver.cc:460] Webserver started at http://127.12.132.194:40647/ using document root <none> and password file <none>
I20260620 08:13:10.753314 30040 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:10.753383 30040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:10.753530 30040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:10.754587 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "d7b1a09a06614ed5bca16ae94eaf7c45"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "55cd8c53e2fad7e145f1231f6f24f4c5"
server_key_iv: "42306b8ffda16ef2f3d42edfd9159608"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.754957 30040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "d7b1a09a06614ed5bca16ae94eaf7c45"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "55cd8c53e2fad7e145f1231f6f24f4c5"
server_key_iv: "42306b8ffda16ef2f3d42edfd9159608"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.756299 30040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:10.757169 30055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.757339 30040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:10.757449 30040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "d7b1a09a06614ed5bca16ae94eaf7c45"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "55cd8c53e2fad7e145f1231f6f24f4c5"
server_key_iv: "42306b8ffda16ef2f3d42edfd9159608"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.757541 30040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:10.766464 30040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:10.768402 30040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:10.768592 30040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:10.768808 30040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:10.769114 30040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:10.769170 30040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.769212 30040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:10.769238 30040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.775456 30040 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:46479
I20260620 08:13:10.775532 30168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:46479 every 8 connection(s)
I20260620 08:13:10.775853 30040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:13:10.778285 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30040
I20260620 08:13:10.778429 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:10.778650 12819 external_mini_cluster.cc:1428] Setting key 7fe7a679c8d0fdcb6fdb0935450edeef
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:10.786522 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.776581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:51985 (local address 127.12.132.254:42877)
0620 08:13:10.776626 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.776627 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.776821 (+ 194us) server_negotiation.cc:407] Connection header received
0620 08:13:10.777298 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.777300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.777310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.777375 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:10.778356 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.779117 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.780146 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.780456 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.780552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.783218 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:10.783219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:10.783220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:10.783227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:10.785043 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.785532 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.785533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.785534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.785595 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.785873 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.785873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.785874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.785985 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:10.786065 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.786349 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.786431 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:10.786710 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.776569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42877 (local address 127.12.132.194:51985)
0620 08:13:10.776704 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.776737 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.777193 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.777422 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.777425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.777964 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:10.778231 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.778237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.779252 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.779253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.780055 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.780057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.780599 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.780600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.780776 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.781285 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:10.781296 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:10.783102 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:10.785152 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.785160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.785162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.785451 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.785669 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.785670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.785670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.785792 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.786056 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:10.786063 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:10.786202 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.786316 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.786476 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260620 08:13:10.787310 30169 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42877
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
I20260620 08:13:10.787429 30169 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:10.787636 30169 heartbeater.cc:511] Master 127.12.132.254:42877 requested a full tablet report, sending...
I20260620 08:13:10.788312 29844 ts_manager.cc:194] Registered new tserver with Master: d7b1a09a06614ed5bca16ae94eaf7c45 (127.12.132.194:46479)
I20260620 08:13:10.788754 29844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:51985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:13:10.807408 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:10.890506 30174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:10.890722 30174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:10.890753 30174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:10.891882 30174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:10.891922 30174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:10.891948 30174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:10.893479 30174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:42877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:10.893709 30174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:10.893885 30174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:10.899081 30182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:10.899087 30180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:10.899098 30179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:10.899140 30174 server_base.cc:1034] running on GCE node
I20260620 08:13:10.899461 30174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:10.899698 30174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:10.900892 30174 hybrid_clock.cc:648] HybridClock initialized: now 1781943190900861 us; error 41 us; skew 500 ppm
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:10.903784 30174 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:13:10.904393 30174 webserver.cc:460] Webserver started at http://127.12.132.195:42045/ using document root <none> and password file <none>
I20260620 08:13:10.904754 30174 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:10.904836 30174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:10.905009 30174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:10.906240 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "f05e81df7e18433b8b3878bf1df07aa7"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "c1534f63d2287d030e4fdd909f00a73a"
server_key_iv: "4090c7ce3529fe52e8219dcc6b2b5058"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.906666 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "f05e81df7e18433b8b3878bf1df07aa7"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "c1534f63d2287d030e4fdd909f00a73a"
server_key_iv: "4090c7ce3529fe52e8219dcc6b2b5058"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.908274 30174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:10.909410 30189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.909809 30174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:10.909976 30174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "f05e81df7e18433b8b3878bf1df07aa7"
format_stamp: "Formatted at 2026-06-20 08:13:10 on dist-test-slave-z0sp"
server_key: "c1534f63d2287d030e4fdd909f00a73a"
server_key_iv: "4090c7ce3529fe52e8219dcc6b2b5058"
server_key_version: "encryptionkey@0"
I20260620 08:13:10.910079 30174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:10.922732 30174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:10.925299 30174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:10.925475 30174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:10.925758 30174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:10.926111 30174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:10.926172 30174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.926214 30174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:10.926239 30174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:10.934347 30174 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:38645
I20260620 08:13:10.934401 30302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:38645 every 8 connection(s)
I20260620 08:13:10.934895 30174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:13:10 dist-test-slave-z0sp krb5kdc[29799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943190, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:10.942926 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30174
I20260620 08:13:10.943032 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:10.943181 12819 external_mini_cluster.cc:1428] Setting key eb796549f80257292465f7bab52a8d10
I20260620 08:13:10.945401 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.936040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:41921 (local address 127.12.132.254:42877)
0620 08:13:10.936121 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.936123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.936339 (+ 216us) server_negotiation.cc:407] Connection header received
0620 08:13:10.936927 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.936928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.936938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.937002 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:10.937906 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.938396 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.939438 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.939683 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.939775 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.942024 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:10.942026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:10.942026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:10.942036 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:10.943779 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.944454 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.944455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.944456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.944558 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.944876 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.944877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.944877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.944983 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:10.945036 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.945274 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.945339 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:13:10.945586 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.936055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:42877 (local address 127.12.132.195:41921)
0620 08:13:10.936222 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.936259 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.936832 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.937053 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.937057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.937621 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:10.937813 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.937817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.938498 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.938499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.939278 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.939280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.939799 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.939800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.939908 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.940371 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:10.940379 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:10.941848 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:10.943907 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.943920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.943923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.944383 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.944676 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:10.944678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:10.944678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:10.944798 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:10.945046 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:10.945050 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:10.945159 (+ 109us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.945243 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.945373 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260620 08:13:10.946094 30303 heartbeater.cc:347] Connected to a master server at 127.12.132.254:42877
I20260620 08:13:10.946204 30303 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:10.946412 30303 heartbeater.cc:511] Master 127.12.132.254:42877 requested a full tablet report, sending...
I20260620 08:13:10.946784 29844 ts_manager.cc:194] Registered new tserver with Master: f05e81df7e18433b8b3878bf1df07aa7 (127.12.132.195:38645)
I20260620 08:13:10.947117 29844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:41921
I20260620 08:13:10.955497 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:10.963977 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.956860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.12.132.254:42877)
0620 08:13:10.956932 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.956934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.957004 (+ 70us) server_negotiation.cc:407] Connection header received
0620 08:13:10.957080 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.957081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.957091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.957162 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:10.957708 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.958206 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.958939 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.959134 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.959210 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.960669 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:10.960671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:10.960671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:10.960678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:10.962356 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.962750 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.962751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.962752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.962912 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:10.963270 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:10.963271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:10.963271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:10.963469 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:10.963595 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.963749 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.963856 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 08:13:10.966114 29844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:10.967696 29844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:10.974792 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:33151 (local address 127.0.0.1:50192)
0620 08:13:10.970790 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.970802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.970859 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.971316 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.971317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.971319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:10.971390 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.971392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972411 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.972412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.973704 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.973707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.974545 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.974546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.974622 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.974633 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.974699 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.974738 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":1,"thread_start_us":154,"threads_started":1}
I20260620 08:13:10.974875 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.12.132.193:33151)
0620 08:13:10.971136 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.971141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.971152 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:10.971194 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.971197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.971214 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.971303 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:10.971454 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972266 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.973804 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.974430 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.974511 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.974564 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.974736 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.974775 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260620 08:13:10.975641 29969 tablet_service.cc:1467] Processing CreateTablet for tablet a849c2727af24ecbb35dc16978a79193 (DEFAULT_TABLE table=test-table [id=b5ddf98574f8429587c58868764f34ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:10.976013 29969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a849c2727af24ecbb35dc16978a79193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:10.976017 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:38645 (local address 127.0.0.1:34210)
0620 08:13:10.970492 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.970529 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.970596 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.971173 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.971176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.971177 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:10.971278 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.971280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972675 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.972676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.973996 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.974000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.975750 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.975751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.975825 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.975829 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.975897 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.975937 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":58,"thread_start_us":70,"threads_started":1}
I20260620 08:13:10.976116 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:46479 (local address 127.0.0.1:37306)
0620 08:13:10.970512 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:10.970541 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:10.970598 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:10.971171 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:10.971174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:10.971177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:10.971278 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.971281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972685 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.972685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.974017 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:10.974019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.975836 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.975838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:10.975917 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.975921 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:10.976007 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 08:13:10.976049 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":56,"thread_start_us":84,"threads_started":1}
I20260620 08:13:10.976140 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.12.132.194:46479)
0620 08:13:10.970984 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.970988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.970998 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:10.971034 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.971035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.971049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.971356 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:10.971421 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972561 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.974107 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.974822 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.975831 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.975886 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.976022 (+ 136us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.976062 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":108,"threads_started":1}
I20260620 08:13:10.976297 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:10.970555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.12.132.195:38645)
0620 08:13:10.970974 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:10.970978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:10.970990 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:10.971030 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:10.971032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:10.971079 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:10.971356 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:10.971422 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.972548 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.974914 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:10.975663 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:10.975722 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:10.975753 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:10.976180 (+ 427us) server_negotiation.cc:299] Negotiation successful
0620 08:13:10.976224 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":81,"threads_started":1}
I20260620 08:13:10.976789 30103 tablet_service.cc:1467] Processing CreateTablet for tablet a849c2727af24ecbb35dc16978a79193 (DEFAULT_TABLE table=test-table [id=b5ddf98574f8429587c58868764f34ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:10.977090 30103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a849c2727af24ecbb35dc16978a79193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:10.979413 30320 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Bootstrap starting.
I20260620 08:13:10.980003 30320 tablet_bootstrap.cc:654] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:10.980254 30321 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Bootstrap starting.
I20260620 08:13:10.980625 30320 log.cc:824] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:10.980665 30237 tablet_service.cc:1467] Processing CreateTablet for tablet a849c2727af24ecbb35dc16978a79193 (DEFAULT_TABLE table=test-table [id=b5ddf98574f8429587c58868764f34ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:10.980947 30321 tablet_bootstrap.cc:654] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:10.980986 30237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a849c2727af24ecbb35dc16978a79193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:10.981510 30321 log.cc:824] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:10.982353 30321 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: No bootstrap required, opened a new log
I20260620 08:13:10.982441 30321 ts_tablet_manager.cc:1397] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:10.984172 30324 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Bootstrap starting.
I20260620 08:13:10.984510 30321 raft_consensus.cc:348] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.984691 30321 raft_consensus.cc:374] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:10.984731 30321 raft_consensus.cc:729] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b1a09a06614ed5bca16ae94eaf7c45, State: Initialized, Role: FOLLOWER
I20260620 08:13:10.984776 30320 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: No bootstrap required, opened a new log
I20260620 08:13:10.984828 30320 ts_tablet_manager.cc:1397] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260620 08:13:10.984827 30324 tablet_bootstrap.cc:654] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:10.984864 30321 consensus_queue.cc:260] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.985144 30321 ts_tablet_manager.cc:1428] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:13:10.985205 30169 heartbeater.cc:503] Master 127.12.132.254:42877 was elected leader, sending a full tablet report...
I20260620 08:13:10.985349 30324 log.cc:824] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:10.986191 30324 tablet_bootstrap.cc:492] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: No bootstrap required, opened a new log
I20260620 08:13:10.986253 30324 ts_tablet_manager.cc:1397] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:13:10.986804 30320 raft_consensus.cc:348] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.986931 30320 raft_consensus.cc:374] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:10.986953 30320 raft_consensus.cc:729] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb01bdd45f44b1c81b1a71e758d3c80, State: Initialized, Role: FOLLOWER
I20260620 08:13:10.987056 30320 consensus_queue.cc:260] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.987269 30320 ts_tablet_manager.cc:1428] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 08:13:10.987318 30035 heartbeater.cc:503] Master 127.12.132.254:42877 was elected leader, sending a full tablet report...
I20260620 08:13:10.988095 30324 raft_consensus.cc:348] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.988289 30324 raft_consensus.cc:374] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:10.988335 30324 raft_consensus.cc:729] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05e81df7e18433b8b3878bf1df07aa7, State: Initialized, Role: FOLLOWER
I20260620 08:13:10.988425 30324 consensus_queue.cc:260] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:10.988698 30324 ts_tablet_manager.cc:1428] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:13:10.988710 30303 heartbeater.cc:503] Master 127.12.132.254:42877 was elected leader, sending a full tablet report...
W20260620 08:13:11.026432 30170 tablet.cc:2369] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:13:11.112350 30036 tablet.cc:2369] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:13:11.185612 30304 tablet.cc:2369] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:11.341032 30328 raft_consensus.cc:482] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:11.341161 30328 raft_consensus.cc:504] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:11.341624 30328 leader_election.cc:288] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7b1a09a06614ed5bca16ae94eaf7c45 (127.12.132.194:46479), 4cb01bdd45f44b1c81b1a71e758d3c80 (127.12.132.193:33151)
I20260620 08:13:11.344691 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.341863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:46479 (local address 127.12.132.195:51405)
0620 08:13:11.341923 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:11.341936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:11.341984 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:11.342311 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:11.342313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:11.342314 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:11.342395 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.342397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.343077 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.343078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.343872 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.343874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.344423 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.344423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.344531 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.344536 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:13:11.344587 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 08:13:11.344622 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":17}
I20260620 08:13:11.344688 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.342118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:51405 (local address 127.12.132.194:46479)
0620 08:13:11.342188 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.342190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.342201 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:11.342238 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.342239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.342247 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.342307 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:11.342477 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.342988 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.343972 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.344333 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.344426 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.344434 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.344592 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.344624 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 08:13:11.344882 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.342150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:58631 (local address 127.12.132.193:33151)
0620 08:13:11.342240 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.342243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.342254 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:11.342282 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.342283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.342292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.342346 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:11.342574 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.343053 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.344009 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.344446 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.344608 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.344617 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.344783 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.344819 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 08:13:11.344890 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.342007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:33151 (local address 127.12.132.195:58631)
0620 08:13:11.342162 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:11.342173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:11.342220 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:11.342385 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:11.342386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:11.342389 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:11.342494 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.342496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.343140 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.343141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.343881 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.343883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.344614 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.344615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.344700 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.344703 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:11.344777 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 08:13:11.344815 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260620 08:13:11.345278 29989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a849c2727af24ecbb35dc16978a79193" candidate_uuid: "f05e81df7e18433b8b3878bf1df07aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" is_pre_election: true
I20260620 08:13:11.345290 30123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a849c2727af24ecbb35dc16978a79193" candidate_uuid: "f05e81df7e18433b8b3878bf1df07aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" is_pre_election: true
I20260620 08:13:11.345391 29989 raft_consensus.cc:2454] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e81df7e18433b8b3878bf1df07aa7 in term 0.
I20260620 08:13:11.345444 30123 raft_consensus.cc:2454] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f05e81df7e18433b8b3878bf1df07aa7 in term 0.
I20260620 08:13:11.345614 30191 leader_election.cc:302] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb01bdd45f44b1c81b1a71e758d3c80, f05e81df7e18433b8b3878bf1df07aa7; no voters:
I20260620 08:13:11.345757 30328 raft_consensus.cc:2781] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:11.345815 30328 raft_consensus.cc:482] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:11.345844 30328 raft_consensus.cc:3037] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:11.346513 30328 raft_consensus.cc:504] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:11.346642 30328 leader_election.cc:288] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [CANDIDATE]: Term 1 election: Requested vote from peers d7b1a09a06614ed5bca16ae94eaf7c45 (127.12.132.194:46479), 4cb01bdd45f44b1c81b1a71e758d3c80 (127.12.132.193:33151)
I20260620 08:13:11.346838 30123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a849c2727af24ecbb35dc16978a79193" candidate_uuid: "f05e81df7e18433b8b3878bf1df07aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45"
I20260620 08:13:11.346856 29989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a849c2727af24ecbb35dc16978a79193" candidate_uuid: "f05e81df7e18433b8b3878bf1df07aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80"
I20260620 08:13:11.346928 29989 raft_consensus.cc:3037] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:11.346928 30123 raft_consensus.cc:3037] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:11.347798 29989 raft_consensus.cc:2454] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e81df7e18433b8b3878bf1df07aa7 in term 1.
I20260620 08:13:11.347798 30123 raft_consensus.cc:2454] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f05e81df7e18433b8b3878bf1df07aa7 in term 1.
I20260620 08:13:11.348021 30191 leader_election.cc:302] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb01bdd45f44b1c81b1a71e758d3c80, f05e81df7e18433b8b3878bf1df07aa7; no voters:
I20260620 08:13:11.348115 30328 raft_consensus.cc:2781] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:11.348272 30328 raft_consensus.cc:686] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 1 LEADER]: Becoming Leader. State: Replica: f05e81df7e18433b8b3878bf1df07aa7, State: Running, Role: LEADER
I20260620 08:13:11.348364 30328 consensus_queue.cc:237] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } }
I20260620 08:13:11.349179 29844 catalog_manager.cc:5495] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f05e81df7e18433b8b3878bf1df07aa7 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "f05e81df7e18433b8b3878bf1df07aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f05e81df7e18433b8b3878bf1df07aa7" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 38645 } health_report { overall_health: HEALTHY } } }
I20260620 08:13:11.416723 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.413860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.12.132.195:38645)
0620 08:13:11.413961 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.413963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.413986 (+ 23us) server_negotiation.cc:407] Connection header received
0620 08:13:11.414030 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.414031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.414047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.414110 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:11.414230 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.414708 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.415678 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.415919 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.416108 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.416270 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:11.416372 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:11.416445 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.416626 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.416650 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 08:13:11.418682 29989 raft_consensus.cc:1261] T a849c2727af24ecbb35dc16978a79193 P 4cb01bdd45f44b1c81b1a71e758d3c80 [term 1 FOLLOWER]: Refusing update from remote peer f05e81df7e18433b8b3878bf1df07aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:11.418682 30123 raft_consensus.cc:1261] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 1 FOLLOWER]: Refusing update from remote peer f05e81df7e18433b8b3878bf1df07aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:11.419062 30328 consensus_queue.cc:1035] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b1a09a06614ed5bca16ae94eaf7c45" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:11.419200 30330 consensus_queue.cc:1035] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb01bdd45f44b1c81b1a71e758d3c80" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 33151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:11.423048 30333 mvcc.cc:205] Tried to move back new op lower bound from 7298839312047865856 to 7298839311763566592. Current Snapshot: MvccSnapshot[applied={T|T < 7298839312047865856 or (T in {7298839312047865856})}]
I20260620 08:13:11.422866 30334 mvcc.cc:205] Tried to move back new op lower bound from 7298839312047865856 to 7298839311763566592. Current Snapshot: MvccSnapshot[applied={T|T < 7298839312047865856}]
I20260620 08:13:11.425169 29843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:11.425244 29843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:11.426034 29843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87dd56e76cbf47a2a1820ab20da574ae: Sending DeleteTablet for 3 replicas of tablet a849c2727af24ecbb35dc16978a79193
I20260620 08:13:11.426318 30103 tablet_service.cc:1514] Processing DeleteTablet for tablet a849c2727af24ecbb35dc16978a79193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:11 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:37306
I20260620 08:13:11.426368 30237 tablet_service.cc:1514] Processing DeleteTablet for tablet a849c2727af24ecbb35dc16978a79193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:11 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:34210
I20260620 08:13:11.426398 29969 tablet_service.cc:1514] Processing DeleteTablet for tablet a849c2727af24ecbb35dc16978a79193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:11 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:50192
I20260620 08:13:11.426590 30342 tablet_replica.cc:330] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: stopping tablet replica
I20260620 08:13:11.426695 30342 raft_consensus.cc:2229] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:11.426848 30342 raft_consensus.cc:2258] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:11.426877 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29906
I20260620 08:13:11.429093 30342 ts_tablet_manager.cc:1905] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:11.431062 30342 ts_tablet_manager.cc:1918] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:11.431173 30342 log.cc:1196] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/a849c2727af24ecbb35dc16978a79193
I20260620 08:13:11.431588 30342 ts_tablet_manager.cc:1939] T a849c2727af24ecbb35dc16978a79193 P d7b1a09a06614ed5bca16ae94eaf7c45: Deleting consensus metadata
I20260620 08:13:11.432029 29829 catalog_manager.cc:4841] TS d7b1a09a06614ed5bca16ae94eaf7c45 (127.12.132.194:46479): tablet a849c2727af24ecbb35dc16978a79193 (table test-table [id=b5ddf98574f8429587c58868764f34ab]) successfully deleted
W20260620 08:13:11.433446 29829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:33151 (error 108)
I20260620 08:13:11.433681 30343 tablet_replica.cc:330] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: stopping tablet replica
I20260620 08:13:11.433971 30343 raft_consensus.cc:2229] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:11.434087 30343 raft_consensus.cc:2258] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 08:13:11.434343 30315 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:11.433843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:33151 (local address 127.0.0.1:50196)
0620 08:13:11.433898 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:11.433911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:11.433953 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:11.434300 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:33151: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 08:13:11.434456 29829 catalog_manager.cc:4584] TS 4cb01bdd45f44b1c81b1a71e758d3c80 (127.12.132.193:33151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a849c2727af24ecbb35dc16978a79193: Network error: Client connection negotiation failed: client connection to 127.12.132.193:33151: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 08:13:11.434556 30343 ts_tablet_manager.cc:1905] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:11.434741 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30040
I20260620 08:13:11.438166 30343 ts_tablet_manager.cc:1918] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:11.438238 30343 log.cc:1196] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/a849c2727af24ecbb35dc16978a79193
I20260620 08:13:11.438491 30343 ts_tablet_manager.cc:1939] T a849c2727af24ecbb35dc16978a79193 P f05e81df7e18433b8b3878bf1df07aa7: Deleting consensus metadata
I20260620 08:13:11.438918 29828 catalog_manager.cc:4841] TS f05e81df7e18433b8b3878bf1df07aa7 (127.12.132.195:38645): tablet a849c2727af24ecbb35dc16978a79193 (table test-table [id=b5ddf98574f8429587c58868764f34ab]) successfully deleted
I20260620 08:13:11.441570 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30174
I20260620 08:13:11.446671 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 29812
2026-06-20T08:13:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2352 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T08:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:11Z Disabled control of system clock
I20260620 08:13:11.476897 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:46007
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:46007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:11.559839 30349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:11.560036 30349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:11.560057 30349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:11.561116 30349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:11.561151 30349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:11.561164 30349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:11.561177 30349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:11.561188 30349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:11.562525 30349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:46007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:46007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:11.562705 30349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:11.562922 30349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:11.568053 30357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.568032 30355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.568003 30354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:11.567986 30349 server_base.cc:1034] running on GCE node
I20260620 08:13:11.568526 30349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:11.568823 30349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:11.569999 30349 hybrid_clock.cc:648] HybridClock initialized: now 1781943191569987 us; error 31 us; skew 500 ppm
I20260620 08:13:11.579929 30349 webserver.cc:460] Webserver started at http://127.12.132.254:44727/ using document root <none> and password file <none>
I20260620 08:13:11.580281 30349 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:11.580343 30349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:11.580574 30349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:11.581920 30349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "d188b65ade0945a2bb7ca8d3273e27ff"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "b24c02ce5be298a564b099edceec38e3"
server_key_iv: "79d823d150a887f6a3da048377a3414b"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.582377 30349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "d188b65ade0945a2bb7ca8d3273e27ff"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "b24c02ce5be298a564b099edceec38e3"
server_key_iv: "79d823d150a887f6a3da048377a3414b"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.584151 30349 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260620 08:13:11.585470 30363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.585700 30349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:11.585788 30349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "d188b65ade0945a2bb7ca8d3273e27ff"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "b24c02ce5be298a564b099edceec38e3"
server_key_iv: "79d823d150a887f6a3da048377a3414b"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.585862 30349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:11.594322 30349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:11.594554 30349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:11.594648 30349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:11.598467 30349 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:46007
I20260620 08:13:11.598562 30415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:46007 every 8 connection(s)
I20260620 08:13:11.599198 30349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:11.600767 30416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:11.602496 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30349
I20260620 08:13:11.602608 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:11.602898 12819 external_mini_cluster.cc:1428] Setting key 986628e471c8b28f4e9ab3c7e4c612c9
I20260620 08:13:11.603536 30416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: Bootstrap starting.
I20260620 08:13:11.604190 30416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:11.604552 30416 log.cc:824] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:11.605688 30416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: No bootstrap required, opened a new log
I20260620 08:13:11.608079 30416 raft_consensus.cc:348] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } }
I20260620 08:13:11.608247 30416 raft_consensus.cc:374] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:11.608286 30416 raft_consensus.cc:729] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d188b65ade0945a2bb7ca8d3273e27ff, State: Initialized, Role: FOLLOWER
I20260620 08:13:11.608424 30416 consensus_queue.cc:260] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } }
I20260620 08:13:11.608537 30416 raft_consensus.cc:388] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:11.608587 30416 raft_consensus.cc:482] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:11.608645 30416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:11.609395 30416 raft_consensus.cc:504] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } }
I20260620 08:13:11.609504 30416 leader_election.cc:302] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d188b65ade0945a2bb7ca8d3273e27ff; no voters:
I20260620 08:13:11.609686 30416 leader_election.cc:288] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:11.609802 30421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:11.609977 30416 sys_catalog.cc:564] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:11.610036 30421 raft_consensus.cc:686] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [term 1 LEADER]: Becoming Leader. State: Replica: d188b65ade0945a2bb7ca8d3273e27ff, State: Running, Role: LEADER
I20260620 08:13:11.610051 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.603649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.12.132.254:46007)
0620 08:13:11.603881 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.603890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.603921 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:13:11.604671 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.604684 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.604760 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.604903 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:11.605219 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.606829 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.607644 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.608310 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.608799 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.609214 (+ 415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:11.609218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:11.609218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:11.609423 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:11.609466 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:11.609480 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:11.609584 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.609769 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.609864 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260620 08:13:11.610163 30421 consensus_queue.cc:237] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } }
I20260620 08:13:11.610457 30421 sys_catalog.cc:455] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } } }
I20260620 08:13:11.610526 30421 sys_catalog.cc:458] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:11.610659 30421 sys_catalog.cc:455] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d188b65ade0945a2bb7ca8d3273e27ff. Latest consensus state: current_term: 1 leader_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d188b65ade0945a2bb7ca8d3273e27ff" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 46007 } } }
I20260620 08:13:11.610751 30421 sys_catalog.cc:458] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:11.610828 30429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:11.611331 30429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:11.612824 30429 catalog_manager.cc:1347] Generated new cluster ID: 54739095cf884827a3331fcc29b2c1e4
I20260620 08:13:11.612877 30429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:11.627566 30429 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:11.628121 30429 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:11.631573 30429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: Generated new TSK 0
I20260620 08:13:11.631850 30429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:11.640825 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--builtin_ntp_servers=127.12.132.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:11.728717 30440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:11.728901 30440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:11.728922 30440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:11.730041 30440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:11.730077 30440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:11.730103 30440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:11.731418 30440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:11.731648 30440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:11.731997 30440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:11.738444 30446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.738440 30448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.738458 30445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:11.738976 30440 server_base.cc:1034] running on GCE node
I20260620 08:13:11.739238 30440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:11.739675 30440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:11.740897 30440 hybrid_clock.cc:648] HybridClock initialized: now 1781943191740877 us; error 42 us; skew 500 ppm
I20260620 08:13:11.742611 30440 webserver.cc:460] Webserver started at http://127.12.132.193:39759/ using document root <none> and password file <none>
I20260620 08:13:11.742901 30440 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:11.742959 30440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:11.743108 30440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:11.744359 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "f41bde57065841df8a2bc8005db385a8"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "67f11b105367a5a52f0591d76412fdd1"
server_key_iv: "e40e98f1380302fd7d9a603072acbfe8"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.744838 30440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "f41bde57065841df8a2bc8005db385a8"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "67f11b105367a5a52f0591d76412fdd1"
server_key_iv: "e40e98f1380302fd7d9a603072acbfe8"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.746448 30440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:11.747526 30454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.747756 30440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:11.747846 30440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "f41bde57065841df8a2bc8005db385a8"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "67f11b105367a5a52f0591d76412fdd1"
server_key_iv: "e40e98f1380302fd7d9a603072acbfe8"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.747918 30440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:11.761952 30440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:11.762199 30440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:11.762305 30440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:11.762486 30440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:11.762750 30440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:11.762780 30440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.762800 30440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:11.762813 30440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.768767 30440 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:36109
I20260620 08:13:11.768849 30567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:36109 every 8 connection(s)
I20260620 08:13:11.769244 30440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:13:11.774556 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.770026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52075 (local address 127.12.132.254:46007)
0620 08:13:11.770125 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.770127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.770336 (+ 209us) server_negotiation.cc:407] Connection header received
0620 08:13:11.770910 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.770911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.770922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.770970 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:11.771637 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.772134 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.773210 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.773419 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.773502 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.774278 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:11.774279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:11.774279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:11.774284 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:11.774298 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:11.774302 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:11.774352 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.774456 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.774480 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:13:11.774816 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.769994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46007 (local address 127.12.132.193:52075)
0620 08:13:11.770218 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:11.770250 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:11.770795 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:11.771050 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:11.771053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:11.771060 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:11.771531 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.771535 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.772258 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.772258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.773111 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.773114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.773513 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.773514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.773636 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.774122 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:11.774123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:11.774148 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:11.774169 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:11.774367 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:11.774374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:11.774381 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 08:13:11.774459 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 08:13:11.774554 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260620 08:13:11.775326 30568 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46007
I20260620 08:13:11.775398 30568 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:11.775622 30568 heartbeater.cc:511] Master 127.12.132.254:46007 requested a full tablet report, sending...
I20260620 08:13:11.776011 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30440
I20260620 08:13:11.776114 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:11.776232 30380 ts_manager.cc:194] Registered new tserver with Master: f41bde57065841df8a2bc8005db385a8 (127.12.132.193:36109)
I20260620 08:13:11.776346 12819 external_mini_cluster.cc:1428] Setting key 4ddb313a794d8f8f052fbbfd4e38d7fb
I20260620 08:13:11.777146 30380 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.193:52075
I20260620 08:13:11.777626 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--builtin_ntp_servers=127.12.132.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:11.862401 30571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:11.862596 30571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:11.862617 30571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:11.863684 30571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:11.863719 30571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:11.863744 30571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:11.865010 30571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:11.865176 30571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:11.865394 30571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:13:11.871294 30571 server_base.cc:1034] running on GCE node
W20260620 08:13:11.871258 30577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.871258 30576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:11.871244 30579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:11.871593 30571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:11.871833 30571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:11.873040 30571 hybrid_clock.cc:648] HybridClock initialized: now 1781943191873021 us; error 42 us; skew 500 ppm
I20260620 08:13:11.874262 30571 webserver.cc:460] Webserver started at http://127.12.132.194:36473/ using document root <none> and password file <none>
I20260620 08:13:11.874518 30571 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:11.874573 30571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:11.874696 30571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:11.875778 30571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "23630e306bb79527dd5eb4e5a14b31a4"
server_key_iv: "e5f629b54699def69808c83bb8f741c7"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.876148 30571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "23630e306bb79527dd5eb4e5a14b31a4"
server_key_iv: "e5f629b54699def69808c83bb8f741c7"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.877552 30571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:11.878364 30585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.878566 30571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:11.878647 30571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa"
format_stamp: "Formatted at 2026-06-20 08:13:11 on dist-test-slave-z0sp"
server_key: "23630e306bb79527dd5eb4e5a14b31a4"
server_key_iv: "e5f629b54699def69808c83bb8f741c7"
server_key_version: "encryptionkey@0"
I20260620 08:13:11.878701 30571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:11.902539 30571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:11.902859 30571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:11.902968 30571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:11.903148 30571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:11.903468 30571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:11.903528 30571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.903560 30571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:11.903590 30571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:11.909554 30571 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:35015
I20260620 08:13:11.909590 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:35015 every 8 connection(s)
I20260620 08:13:11.909924 30571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:13:11.913038 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30571
I20260620 08:13:11.913131 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:11.913332 12819 external_mini_cluster.cc:1428] Setting key 0949241a419dbf0df7749ecf8b611b8e
I20260620 08:13:11.914336 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--builtin_ntp_servers=127.12.132.212:36637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 08:13:11.915602 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.910742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:55017 (local address 127.12.132.254:46007)
0620 08:13:11.910823 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:11.910825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:11.910979 (+ 154us) server_negotiation.cc:407] Connection header received
0620 08:13:11.911494 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:11.911495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:11.911506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:11.911559 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:11.912164 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.912734 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.913905 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.914186 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.914276 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.915323 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:11.915325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:11.915325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:11.915330 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:11.915352 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:11.915356 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:11.915405 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:11.915501 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 08:13:11.915526 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 08:13:11.915856 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:11.910645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46007 (local address 127.12.132.194:55017)
0620 08:13:11.910867 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:11.910901 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:11.911407 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:11.911607 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:11.911610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:11.911618 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:11.912085 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.912089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.912831 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.912832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.913818 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:11.913821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:11.914288 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:11.914289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:11.914457 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:11.915155 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:11.915155 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:11.915192 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:11.915213 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:11.915406 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:11.915411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:11.915414 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:11.915503 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 08:13:11.915614 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260620 08:13:11.916325 30699 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46007
I20260620 08:13:11.916433 30699 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:11.916643 30699 heartbeater.cc:511] Master 127.12.132.254:46007 requested a full tablet report, sending...
I20260620 08:13:11.917116 30380 ts_manager.cc:194] Registered new tserver with Master: 4bddaa8c96a749e8a2388dd3e2bc2efa (127.12.132.194:35015)
I20260620 08:13:11.917738 30380 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.194:55017
W20260620 08:13:12.005661 30702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:12.005911 30702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:12.005946 30702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:12.007977 30702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:12.008013 30702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:12.008039 30702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:12.009644 30702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:12.009860 30702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:12.010116 30702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:13:12.015478 30702 server_base.cc:1034] running on GCE node
W20260620 08:13:12.015396 30708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:12.015394 30707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:12.015419 30710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:12.015838 30702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:12.016095 30702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:12.017289 30702 hybrid_clock.cc:648] HybridClock initialized: now 1781943192017240 us; error 86 us; skew 500 ppm
I20260620 08:13:12.018563 30702 webserver.cc:460] Webserver started at http://127.12.132.195:45843/ using document root <none> and password file <none>
I20260620 08:13:12.018815 30702 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:12.018893 30702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:12.019047 30702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:12.020109 30702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "f61a0540bcb04c4a9aa3bc104998f0af"
format_stamp: "Formatted at 2026-06-20 08:13:12 on dist-test-slave-z0sp"
server_key: "1e262285db9038c93060d931d6c933dd"
server_key_iv: "b3d8d77225959d751f7b23390d15426f"
server_key_version: "encryptionkey@0"
I20260620 08:13:12.020474 30702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "f61a0540bcb04c4a9aa3bc104998f0af"
format_stamp: "Formatted at 2026-06-20 08:13:12 on dist-test-slave-z0sp"
server_key: "1e262285db9038c93060d931d6c933dd"
server_key_iv: "b3d8d77225959d751f7b23390d15426f"
server_key_version: "encryptionkey@0"
I20260620 08:13:12.021814 30702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:12.022596 30716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:12.022778 30702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:12.022878 30702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "f61a0540bcb04c4a9aa3bc104998f0af"
format_stamp: "Formatted at 2026-06-20 08:13:12 on dist-test-slave-z0sp"
server_key: "1e262285db9038c93060d931d6c933dd"
server_key_iv: "b3d8d77225959d751f7b23390d15426f"
server_key_version: "encryptionkey@0"
I20260620 08:13:12.022962 30702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:12.036062 30702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:12.036363 30702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:12.036577 30702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:12.036813 30702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:12.037159 30702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:12.037218 30702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:12.037261 30702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:12.037295 30702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:12.043157 30702 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:44829
I20260620 08:13:12.043200 30829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:44829 every 8 connection(s)
I20260620 08:13:12.043510 30702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:13:12.045307 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30702
I20260620 08:13:12.045436 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:12.045625 12819 external_mini_cluster.cc:1428] Setting key 340c08aff1ba12e31a4af31bfce319f7
I20260620 08:13:12.048971 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.044191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:53617 (local address 127.12.132.254:46007)
0620 08:13:12.044263 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.044265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.044534 (+ 269us) server_negotiation.cc:407] Connection header received
0620 08:13:12.045036 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.045037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.045047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.045100 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:12.045872 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.046542 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.047512 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.047717 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.047792 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.048693 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:12.048694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:12.048695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:12.048699 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:12.048712 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:12.048716 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:12.048769 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.048867 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.048892 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:13:12.049197 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.044191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:46007 (local address 127.12.132.195:53617)
0620 08:13:12.044373 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.044405 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.044951 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.045170 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.045173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.045181 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:12.045767 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.045780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.046637 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.046638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.047406 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.047408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.047813 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.047814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.047938 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.048486 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:12.048487 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:12.048574 (+ 87us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:12.048595 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:12.048772 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:12.048777 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:12.048780 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.048870 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.048963 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260620 08:13:12.049834 30830 heartbeater.cc:347] Connected to a master server at 127.12.132.254:46007
I20260620 08:13:12.050032 30830 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:12.050361 30830 heartbeater.cc:511] Master 127.12.132.254:46007 requested a full tablet report, sending...
I20260620 08:13:12.050915 30380 ts_manager.cc:194] Registered new tserver with Master: f61a0540bcb04c4a9aa3bc104998f0af (127.12.132.195:44829)
I20260620 08:13:12.051506 30380 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.195:53617
I20260620 08:13:12.057852 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:12.062853 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.059480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.12.132.254:46007)
0620 08:13:12.059612 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.059615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.059635 (+ 20us) server_negotiation.cc:407] Connection header received
0620 08:13:12.059712 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.059712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.059724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.059785 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:12.060055 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.060613 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.061369 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.061556 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.061618 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.062381 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:12.062382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:12.062382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:12.062386 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:12.062399 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:12.062402 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:12.062449 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.062661 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.062696 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260620 08:13:12.064808 30380 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:12.066313 30380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:12.073299 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.069151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:35015 (local address 127.0.0.1:35406)
0620 08:13:12.069444 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.069456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.069511 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.069785 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.069785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.069787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:12.069820 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.069821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.070772 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.070773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.072072 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.072075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.072991 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.072992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.073068 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.073081 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.073156 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.073217 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":74,"threads_started":1}
I20260620 08:13:12.073403 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.069238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.12.132.193:36109)
0620 08:13:12.069448 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.069452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.069462 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:12.069501 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.069504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.069520 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.069603 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:12.069913 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.071049 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.072326 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.073033 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.073129 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.073204 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.073269 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.073306 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":112,"threads_started":1}
I20260620 08:13:12.073520 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.068945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:36109 (local address 127.0.0.1:37808)
0620 08:13:12.069061 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.069098 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.069157 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.069628 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.069631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.069634 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:12.069700 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.069701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.071140 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.071141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.072243 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.072245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.073133 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.073134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.073197 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.073201 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.073255 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.073295 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260620 08:13:12.073736 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.069397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.12.132.194:35015)
0620 08:13:12.069623 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.069627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.069636 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:12.069673 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.069674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.069690 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.069774 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:12.069897 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.070663 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.072160 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.072903 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.072981 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.073074 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.073549 (+ 475us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.073650 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260620 08:13:12.074303 30502 tablet_service.cc:1467] Processing CreateTablet for tablet ab918584ef964c68895c4a2006ad05b8 (DEFAULT_TABLE table=test-table [id=3b88110b4aad4b23932f0f03bc43319a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:12.074529 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.068959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:44829 (local address 127.0.0.1:49588)
0620 08:13:12.069400 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.069410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.069466 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.069693 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.069694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.069696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:12.069776 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.069778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.072313 (+ 2535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.072313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.073137 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.073140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.074254 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.074255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.074327 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.074331 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.074397 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.074462 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":102,"threads_started":1}
I20260620 08:13:12.074551 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.069045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.12.132.195:44829)
0620 08:13:12.069212 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.069216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.069523 (+ 307us) server_negotiation.cc:407] Connection header received
0620 08:13:12.069562 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.069564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.069578 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.069642 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:12.069903 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.071052 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.073224 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.074150 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.074255 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.074336 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.074414 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.074456 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260620 08:13:12.074688 30502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab918584ef964c68895c4a2006ad05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:12.075148 30764 tablet_service.cc:1467] Processing CreateTablet for tablet ab918584ef964c68895c4a2006ad05b8 (DEFAULT_TABLE table=test-table [id=3b88110b4aad4b23932f0f03bc43319a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:12.075431 30764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab918584ef964c68895c4a2006ad05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:12.077598 30844 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: Bootstrap starting.
I20260620 08:13:12.078058 30845 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Bootstrap starting.
I20260620 08:13:12.078054 30633 tablet_service.cc:1467] Processing CreateTablet for tablet ab918584ef964c68895c4a2006ad05b8 (DEFAULT_TABLE table=test-table [id=3b88110b4aad4b23932f0f03bc43319a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:12.078243 30844 tablet_bootstrap.cc:654] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:12.078378 30633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab918584ef964c68895c4a2006ad05b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:12.078567 30844 log.cc:824] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:12.078811 30845 tablet_bootstrap.cc:654] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:12.079124 30845 log.cc:824] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:12.079370 30844 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: No bootstrap required, opened a new log
I20260620 08:13:12.079442 30844 ts_tablet_manager.cc:1397] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:12.079825 30845 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: No bootstrap required, opened a new log
I20260620 08:13:12.079916 30845 ts_tablet_manager.cc:1397] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:12.081809 30848 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Bootstrap starting.
I20260620 08:13:12.081964 30845 raft_consensus.cc:348] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.082132 30845 raft_consensus.cc:374] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:12.082175 30845 raft_consensus.cc:729] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f61a0540bcb04c4a9aa3bc104998f0af, State: Initialized, Role: FOLLOWER
I20260620 08:13:12.082325 30845 consensus_queue.cc:260] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.082289 30844 raft_consensus.cc:348] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.082459 30844 raft_consensus.cc:374] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:12.082499 30844 raft_consensus.cc:729] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41bde57065841df8a2bc8005db385a8, State: Initialized, Role: FOLLOWER
I20260620 08:13:12.082578 30845 ts_tablet_manager.cc:1428] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:13:12.082664 30848 tablet_bootstrap.cc:654] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:12.082703 30830 heartbeater.cc:503] Master 127.12.132.254:46007 was elected leader, sending a full tablet report...
I20260620 08:13:12.082674 30844 consensus_queue.cc:260] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.082896 30844 ts_tablet_manager.cc:1428] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 08:13:12.082921 30568 heartbeater.cc:503] Master 127.12.132.254:46007 was elected leader, sending a full tablet report...
I20260620 08:13:12.082962 30848 log.cc:824] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:12.083743 30848 tablet_bootstrap.cc:492] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: No bootstrap required, opened a new log
I20260620 08:13:12.083815 30848 ts_tablet_manager.cc:1397] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:12.086071 30848 raft_consensus.cc:348] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.086220 30848 raft_consensus.cc:374] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:12.086242 30848 raft_consensus.cc:729] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bddaa8c96a749e8a2388dd3e2bc2efa, State: Initialized, Role: FOLLOWER
I20260620 08:13:12.086346 30848 consensus_queue.cc:260] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.086603 30848 ts_tablet_manager.cc:1428] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 08:13:12.086719 30699 heartbeater.cc:503] Master 127.12.132.254:46007 was elected leader, sending a full tablet report...
I20260620 08:13:12.099776 30852 raft_consensus.cc:482] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:12.099898 30852 raft_consensus.cc:504] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.100288 30852 leader_election.cc:288] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41bde57065841df8a2bc8005db385a8 (127.12.132.193:36109), f61a0540bcb04c4a9aa3bc104998f0af (127.12.132.195:44829)
I20260620 08:13:12.103926 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.100474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:36109 (local address 127.12.132.194:38609)
0620 08:13:12.100666 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.100675 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.100724 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.100872 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.100872 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.100874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:12.100966 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.100967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.101610 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.101610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.102855 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.102857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.103661 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.103662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.103728 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.103730 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.103788 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.103828 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":122,"threads_started":1}
I20260620 08:13:12.103964 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.100571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:48255 (local address 127.12.132.195:44829)
0620 08:13:12.100620 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.100623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.100632 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:12.100663 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.100664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.100674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.100734 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:12.100937 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.101447 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.102852 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.103522 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.103594 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.103603 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.103861 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.103896 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:13:12.104082 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.100443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:44829 (local address 127.12.132.194:48255)
0620 08:13:12.100519 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.100530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:12.100572 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:12.100774 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:12.100775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:12.100777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:12.100852 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.100854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.101530 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.101531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.102766 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:12.102769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.103614 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.103615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:12.103725 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.103729 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:12.103796 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:13:12.103831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":47}
I20260620 08:13:12.104391 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.100576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:38609 (local address 127.12.132.193:36109)
0620 08:13:12.100635 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.100637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.100749 (+ 112us) server_negotiation.cc:407] Connection header received
0620 08:13:12.100775 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.100776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.100787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.100850 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:12.101040 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.101526 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.102928 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.103591 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.103670 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.103679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.103827 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.103906 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 08:13:12.104404 30784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab918584ef964c68895c4a2006ad05b8" candidate_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" is_pre_election: true
I20260620 08:13:12.104586 30784 raft_consensus.cc:2454] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bddaa8c96a749e8a2388dd3e2bc2efa in term 0.
I20260620 08:13:12.104841 30589 leader_election.cc:302] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bddaa8c96a749e8a2388dd3e2bc2efa, f61a0540bcb04c4a9aa3bc104998f0af; no voters:
I20260620 08:13:12.104825 30522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab918584ef964c68895c4a2006ad05b8" candidate_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41bde57065841df8a2bc8005db385a8" is_pre_election: true
I20260620 08:13:12.104965 30522 raft_consensus.cc:2454] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bddaa8c96a749e8a2388dd3e2bc2efa in term 0.
I20260620 08:13:12.105129 30852 raft_consensus.cc:2781] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:12.105167 30852 raft_consensus.cc:482] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:12.105183 30852 raft_consensus.cc:3037] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:12.105849 30852 raft_consensus.cc:504] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.105990 30852 leader_election.cc:288] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [CANDIDATE]: Term 1 election: Requested vote from peers f41bde57065841df8a2bc8005db385a8 (127.12.132.193:36109), f61a0540bcb04c4a9aa3bc104998f0af (127.12.132.195:44829)
I20260620 08:13:12.106118 30784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab918584ef964c68895c4a2006ad05b8" candidate_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61a0540bcb04c4a9aa3bc104998f0af"
I20260620 08:13:12.106182 30784 raft_consensus.cc:3037] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:12.106210 30522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab918584ef964c68895c4a2006ad05b8" candidate_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41bde57065841df8a2bc8005db385a8"
I20260620 08:13:12.106333 30522 raft_consensus.cc:3037] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:12.107178 30784 raft_consensus.cc:2454] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bddaa8c96a749e8a2388dd3e2bc2efa in term 1.
I20260620 08:13:12.107504 30522 raft_consensus.cc:2454] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bddaa8c96a749e8a2388dd3e2bc2efa in term 1.
I20260620 08:13:12.107641 30589 leader_election.cc:302] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bddaa8c96a749e8a2388dd3e2bc2efa, f61a0540bcb04c4a9aa3bc104998f0af; no voters:
I20260620 08:13:12.107719 30852 raft_consensus.cc:2781] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:12.107841 30852 raft_consensus.cc:686] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 1 LEADER]: Becoming Leader. State: Replica: 4bddaa8c96a749e8a2388dd3e2bc2efa, State: Running, Role: LEADER
I20260620 08:13:12.107924 30852 consensus_queue.cc:237] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } }
I20260620 08:13:12.108939 30379 catalog_manager.cc:5495] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bddaa8c96a749e8a2388dd3e2bc2efa (127.12.132.194). New cstate: current_term: 1 leader_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bddaa8c96a749e8a2388dd3e2bc2efa" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 35015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 } health_report { overall_health: UNKNOWN } } }
I20260620 08:13:12.122558 30843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.120201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.12.132.194:35015)
0620 08:13:12.120260 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:12.120262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:12.120287 (+ 25us) server_negotiation.cc:407] Connection header received
0620 08:13:12.120324 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:12.120324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:12.120342 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:12.120393 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:12.120562 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.121017 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.121785 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:12.121978 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:12.122039 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:12.122197 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:12.122310 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:12.122372 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:12.122408 (+ 36us) server_negotiation.cc:299] Negotiation successful
0620 08:13:12.122426 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:13:12.124851 30784 raft_consensus.cc:1261] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 1 FOLLOWER]: Refusing update from remote peer 4bddaa8c96a749e8a2388dd3e2bc2efa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:12.124850 30522 raft_consensus.cc:1261] T ab918584ef964c68895c4a2006ad05b8 P f41bde57065841df8a2bc8005db385a8 [term 1 FOLLOWER]: Refusing update from remote peer 4bddaa8c96a749e8a2388dd3e2bc2efa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:12.125212 30852 consensus_queue.cc:1035] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41bde57065841df8a2bc8005db385a8" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:12.125378 30854 consensus_queue.cc:1035] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f61a0540bcb04c4a9aa3bc104998f0af" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 44829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:12.128641 30858 mvcc.cc:205] Tried to move back new op lower bound from 7298839314939736064 to 7298839314874753024. Current Snapshot: MvccSnapshot[applied={T|T < 7298839314939736064}]
I20260620 08:13:12.128823 30859 mvcc.cc:205] Tried to move back new op lower bound from 7298839314939736064 to 7298839314874753024. Current Snapshot: MvccSnapshot[applied={T|T < 7298839314939736064}]
I20260620 08:13:12.129760 30380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:12.129849 30380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:12.130631 30380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d188b65ade0945a2bb7ca8d3273e27ff: Sending DeleteTablet for 3 replicas of tablet ab918584ef964c68895c4a2006ad05b8
I20260620 08:13:12.131016 30502 tablet_service.cc:1514] Processing DeleteTablet for tablet ab918584ef964c68895c4a2006ad05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:12 UTC) from {username='slave'} at 127.0.0.1:37808
I20260620 08:13:12.131250 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30440
I20260620 08:13:12.131256 30764 tablet_service.cc:1514] Processing DeleteTablet for tablet ab918584ef964c68895c4a2006ad05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:12 UTC) from {username='slave'} at 127.0.0.1:49588
I20260620 08:13:12.131448 30633 tablet_service.cc:1514] Processing DeleteTablet for tablet ab918584ef964c68895c4a2006ad05b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:12 UTC) from {username='slave'} at 127.0.0.1:35406
I20260620 08:13:12.132889 30867 tablet_replica.cc:330] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: stopping tablet replica
I20260620 08:13:12.132993 30867 raft_consensus.cc:2229] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:12.133095 30867 raft_consensus.cc:2258] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:12.133526 30867 ts_tablet_manager.cc:1905] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:12.135440 30867 ts_tablet_manager.cc:1918] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:12.135519 30867 log.cc:1196] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/ab918584ef964c68895c4a2006ad05b8
I20260620 08:13:12.135754 30867 ts_tablet_manager.cc:1939] T ab918584ef964c68895c4a2006ad05b8 P f61a0540bcb04c4a9aa3bc104998f0af: Deleting consensus metadata
I20260620 08:13:12.136643 30367 catalog_manager.cc:4841] TS f61a0540bcb04c4a9aa3bc104998f0af (127.12.132.195:44829): tablet ab918584ef964c68895c4a2006ad05b8 (table test-table [id=3b88110b4aad4b23932f0f03bc43319a]) successfully deleted
I20260620 08:13:12.136778 30868 tablet_replica.cc:330] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: stopping tablet replica
I20260620 08:13:12.136883 30868 raft_consensus.cc:2229] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:12.136981 30868 raft_consensus.cc:2258] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:12.137338 30868 ts_tablet_manager.cc:1905] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 08:13:12.138787 30364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:36109 (error 108)
I20260620 08:13:12.139014 30868 ts_tablet_manager.cc:1918] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:12.139097 30868 log.cc:1196] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/ab918584ef964c68895c4a2006ad05b8
I20260620 08:13:12.139364 30868 ts_tablet_manager.cc:1939] T ab918584ef964c68895c4a2006ad05b8 P 4bddaa8c96a749e8a2388dd3e2bc2efa: Deleting consensus metadata
I20260620 08:13:12.139582 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:12.139436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:36109 (local address 127.0.0.1:37820)
0620 08:13:12.139499 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:12.139548 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:36109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 08:13:12.139719 30364 catalog_manager.cc:4584] TS f41bde57065841df8a2bc8005db385a8 (127.12.132.193:36109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab918584ef964c68895c4a2006ad05b8: Network error: Client connection negotiation failed: client connection to 127.12.132.193:36109: connect: Connection refused (error 111)
I20260620 08:13:12.139837 30364 catalog_manager.cc:4841] TS 4bddaa8c96a749e8a2388dd3e2bc2efa (127.12.132.194:35015): tablet ab918584ef964c68895c4a2006ad05b8 (table test-table [id=3b88110b4aad4b23932f0f03bc43319a]) successfully deleted
I20260620 08:13:12.140237 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30571
I20260620 08:13:12.145841 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30702
I20260620 08:13:12.150553 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30349
2026-06-20T08:13:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (704 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:12 dist-test-slave-z0sp krb5kdc[30872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:12 dist-test-slave-z0sp krb5kdc[30872](info): set up 2 sockets
Jun 20 08:13:12 dist-test-slave-z0sp krb5kdc[30872](info): commencing operation
krb5kdc: starting...
W20260620 08:13:13.192907 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:13.312322 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45797
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:13.396879 30885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:13.397156 30885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:13.397207 30885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:13.399295 30885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:13.399400 30885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:13.399430 30885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:13.399464 30885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:13.399485 30885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:13.401216 30885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:45797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:13.401494 30885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:13.401738 30885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:13.407466 30893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.407598 30885 server_base.cc:1034] running on GCE node
W20260620 08:13:13.407455 30890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.407451 30891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.408015 30885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:13.408305 30885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:13.409474 30885 hybrid_clock.cc:648] HybridClock initialized: now 1781943193409463 us; error 61 us; skew 500 ppm
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:13.412263 30885 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:13.413137 30885 webserver.cc:460] Webserver started at http://127.12.132.254:37433/ using document root <none> and password file <none>
I20260620 08:13:13.413422 30885 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:13.413506 30885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:13.413694 30885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:13.415129 30885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "57301ac0532a495f99f96f79eebdc579"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "230ffc3fe196c8815cc00b4ebf6ab0b5"
server_key_iv: "c34439ea8857c247ab9be71f9965a3f3"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.415557 30885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "57301ac0532a495f99f96f79eebdc579"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "230ffc3fe196c8815cc00b4ebf6ab0b5"
server_key_iv: "c34439ea8857c247ab9be71f9965a3f3"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.417021 30885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260620 08:13:13.417780 30900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.417956 30885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:13:13.418062 30885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "57301ac0532a495f99f96f79eebdc579"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "230ffc3fe196c8815cc00b4ebf6ab0b5"
server_key_iv: "c34439ea8857c247ab9be71f9965a3f3"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.418139 30885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:13.430827 30885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:13.433071 30885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:13.433182 30885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:13.437256 30885 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:45797
I20260620 08:13:13.437311 30952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:45797 every 8 connection(s)
I20260620 08:13:13.437626 30885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:13.438298 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30885
I20260620 08:13:13.438422 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:13.438633 12819 external_mini_cluster.cc:1428] Setting key 0925d615cbbce2ab76ea216495409a9f
I20260620 08:13:13.438772 30953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:13.442112 30953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: Bootstrap starting.
I20260620 08:13:13.442790 30953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:13.443125 30953 log.cc:824] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:13.443843 30953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: No bootstrap required, opened a new log
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:13.446368 30953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } }
I20260620 08:13:13.446525 30953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:13.446589 30953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57301ac0532a495f99f96f79eebdc579, State: Initialized, Role: FOLLOWER
I20260620 08:13:13.446722 30953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } }
I20260620 08:13:13.446805 30953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:13.446836 30953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:13.446877 30953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:13.447551 30953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } }
I20260620 08:13:13.447672 30953 leader_election.cc:302] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57301ac0532a495f99f96f79eebdc579; no voters:
I20260620 08:13:13.447832 30953 leader_election.cc:288] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:13.447911 30958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:13.448076 30953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:13.448096 30958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [term 1 LEADER]: Becoming Leader. State: Replica: 57301ac0532a495f99f96f79eebdc579, State: Running, Role: LEADER
I20260620 08:13:13.448303 30958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } }
I20260620 08:13:13.448714 30959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57301ac0532a495f99f96f79eebdc579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } } }
I20260620 08:13:13.448794 30959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:13.449018 30960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57301ac0532a495f99f96f79eebdc579. Latest consensus state: current_term: 1 leader_uuid: "57301ac0532a495f99f96f79eebdc579" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57301ac0532a495f99f96f79eebdc579" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45797 } } }
I20260620 08:13:13.449396 30960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:13.449548 30967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:13.450026 30967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:13.450485 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.439155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.12.132.254:45797)
0620 08:13:13.439341 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.439357 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.439381 (+ 24us) server_negotiation.cc:407] Connection header received
0620 08:13:13.439882 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.439888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.439930 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.440037 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:13.440861 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.441734 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.442895 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.443656 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.443817 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.446335 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:13.446339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:13.446339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:13.446350 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:13.448455 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.448922 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.448925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.448926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.449046 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.449276 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.449277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.449278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.449587 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:13.449746 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.450232 (+ 486us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.450352 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260620 08:13:13.452446 30967 catalog_manager.cc:1347] Generated new cluster ID: c1665a729a9949c2b6b420d1984773f2
I20260620 08:13:13.452528 30967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:13.465436 30967 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:13.465968 30967 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:13.468740 30967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: Generated new TSK 0
I20260620 08:13:13.468924 30967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:13:13.499215 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:13.581761 30979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:13.581943 30979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:13.581965 30979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:13.583003 30979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:13.583036 30979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:13.583061 30979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:13.584530 30979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:13.584703 30979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:13.584867 30979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:13.589991 30984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.590047 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.590070 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.590049 30979 server_base.cc:1034] running on GCE node
I20260620 08:13:13.590487 30979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:13.590821 30979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:13.592064 30979 hybrid_clock.cc:648] HybridClock initialized: now 1781943193592034 us; error 53 us; skew 500 ppm
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:13.595253 30979 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:13:13.595772 30979 webserver.cc:460] Webserver started at http://127.12.132.193:38007/ using document root <none> and password file <none>
I20260620 08:13:13.596045 30979 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:13.596120 30979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:13.596276 30979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:13.597646 30979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "d761ef7a75144ab18c23921f9bd25d79"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "1f800dd7befec025c5c9510e16227b5b"
server_key_iv: "a8d0398d8903ef2a0c62abcfe2d3e22c"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.598033 30979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "d761ef7a75144ab18c23921f9bd25d79"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "1f800dd7befec025c5c9510e16227b5b"
server_key_iv: "a8d0398d8903ef2a0c62abcfe2d3e22c"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.599385 30979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:13:13.600132 30994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.600303 30979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:13:13.600409 30979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "d761ef7a75144ab18c23921f9bd25d79"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "1f800dd7befec025c5c9510e16227b5b"
server_key_iv: "a8d0398d8903ef2a0c62abcfe2d3e22c"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.600519 30979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:13.609110 30979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:13.611079 30979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:13.611238 30979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:13.611462 30979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:13.611769 30979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:13.611827 30979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.611868 30979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:13.611896 30979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.617843 30979 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:40639
I20260620 08:13:13.617897 31107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:40639 every 8 connection(s)
I20260620 08:13:13.618239 30979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:13:13.624421 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 30979
I20260620 08:13:13.624568 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:13.624788 12819 external_mini_cluster.cc:1428] Setting key 35aa27fd94d4ea0fefe37b243c085171
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:13.629182 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.618954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:36979 (local address 127.12.132.254:45797)
0620 08:13:13.619017 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.619019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.619240 (+ 221us) server_negotiation.cc:407] Connection header received
0620 08:13:13.619758 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.619759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.619770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.619830 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:13.620872 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.621499 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.622459 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.622739 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.622842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.625524 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:13.625526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:13.625526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:13.625535 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:13.627223 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.627885 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.627886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.627887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.627950 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.628273 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.628274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.628275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.628427 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:13.628523 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.628970 (+ 447us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.629071 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 08:13:13.629549 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.618938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45797 (local address 127.12.132.193:36979)
0620 08:13:13.619095 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.619135 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.619625 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.619902 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.619906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.620552 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:13.620766 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.620771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.621561 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.621562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.622329 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.622331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.622912 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.622913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.623025 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.623637 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:13.623645 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:13.625408 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:13.627363 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.627378 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.627381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.627755 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.628067 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.628069 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.628069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.628194 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.628598 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:13.628607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:13.628793 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.628916 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.629319 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260620 08:13:13.630249 31108 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45797
I20260620 08:13:13.630362 31108 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:13.630541 31108 heartbeater.cc:511] Master 127.12.132.254:45797 requested a full tablet report, sending...
I20260620 08:13:13.631111 30917 ts_manager.cc:194] Registered new tserver with Master: d761ef7a75144ab18c23921f9bd25d79 (127.12.132.193:40639)
I20260620 08:13:13.631719 30917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:36979
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:13:13.652602 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:13.732481 31113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:13.732699 31113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:13.732722 31113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:13.733772 31113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:13.733815 31113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:13.733841 31113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:13.735183 31113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:13.735368 31113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:13.735523 31113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:13.740758 31119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.740851 31121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.740754 31118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.741039 31113 server_base.cc:1034] running on GCE node
I20260620 08:13:13.741298 31113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:13.741492 31113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:13.742722 31113 hybrid_clock.cc:648] HybridClock initialized: now 1781943193742694 us; error 54 us; skew 500 ppm
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:13.745697 31113 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:13:13.746129 31113 webserver.cc:460] Webserver started at http://127.12.132.194:41099/ using document root <none> and password file <none>
I20260620 08:13:13.746392 31113 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:13.746465 31113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:13.746613 31113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:13.747659 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "c56fdfa60a4649438dfde063fecb46a2"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "bee104bef7fd3e4725d78688e56dcac1"
server_key_iv: "4b01b0f7a96c103630785fca39d9f6eb"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.748054 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "c56fdfa60a4649438dfde063fecb46a2"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "bee104bef7fd3e4725d78688e56dcac1"
server_key_iv: "4b01b0f7a96c103630785fca39d9f6eb"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.749648 31113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:13.750608 31128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.750998 31113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:13.751171 31113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "c56fdfa60a4649438dfde063fecb46a2"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "bee104bef7fd3e4725d78688e56dcac1"
server_key_iv: "4b01b0f7a96c103630785fca39d9f6eb"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.751291 31113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:13.758870 31113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:13.761583 31113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:13.761775 31113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:13.762020 31113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:13.762367 31113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:13.762439 31113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.762482 31113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:13.762511 31113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.770287 31113 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:34987
I20260620 08:13:13.770339 31241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:34987 every 8 connection(s)
I20260620 08:13:13.770740 31113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:13.777886 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31113
I20260620 08:13:13.778033 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:13.778273 12819 external_mini_cluster.cc:1428] Setting key 94cb2e94ddd7146d0ffdaca2cf47e0eb
I20260620 08:13:13.782797 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.771653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:39703 (local address 127.12.132.254:45797)
0620 08:13:13.771793 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.771797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.772076 (+ 279us) server_negotiation.cc:407] Connection header received
0620 08:13:13.772815 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.772817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.772830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.772908 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:13.773981 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.774641 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.775690 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.775894 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.776052 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.778692 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:13.778694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:13.778694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:13.778703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:13.780895 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.781581 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.781582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.781583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.781669 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.781995 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.781996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.781997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.782143 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:13.782237 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.782570 (+ 333us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.782683 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260620 08:13:13.783039 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.771672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45797 (local address 127.12.132.194:39703)
0620 08:13:13.771932 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.771973 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.772659 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.773038 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.773043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.773627 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:13.773836 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.773841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.774747 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.774748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.775567 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.775569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.776064 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.776065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.776172 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.776803 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:13.776811 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:13.778603 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:13.781032 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.781041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.781043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.781450 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.781766 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.781768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.781768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.781907 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.782224 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:13.782232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:13.782401 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.782545 (+ 144us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.782759 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260620 08:13:13.783849 31242 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45797
I20260620 08:13:13.783969 31242 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:13.784188 31242 heartbeater.cc:511] Master 127.12.132.254:45797 requested a full tablet report, sending...
I20260620 08:13:13.784633 30917 ts_manager.cc:194] Registered new tserver with Master: c56fdfa60a4649438dfde063fecb46a2 (127.12.132.194:34987)
I20260620 08:13:13.785025 30917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:39703
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:13:13.808521 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:13.893249 31247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:13.893465 31247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:13.893488 31247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:13.894544 31247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:13.894579 31247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:13.894605 31247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:13.895953 31247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:45797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:13.896131 31247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:13.896299 31247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:13.901314 31255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.901476 31247 server_base.cc:1034] running on GCE node
W20260620 08:13:13.901575 31253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:13.901728 31252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:13.901973 31247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:13.902295 31247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:13.903467 31247 hybrid_clock.cc:648] HybridClock initialized: now 1781943193903447 us; error 48 us; skew 500 ppm
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:13.906159 31247 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:13:13.906664 31247 webserver.cc:460] Webserver started at http://127.12.132.195:34931/ using document root <none> and password file <none>
I20260620 08:13:13.906905 31247 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:13.906976 31247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:13.907136 31247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:13.908165 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "1e9a8f78ee0c4ff48534ea2354523861"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "7655ba822544c31cafe2f0fb5798118d"
server_key_iv: "76bcefcb536ffe39073ece712d4f8c2b"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.908609 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "1e9a8f78ee0c4ff48534ea2354523861"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "7655ba822544c31cafe2f0fb5798118d"
server_key_iv: "76bcefcb536ffe39073ece712d4f8c2b"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.909927 31247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:13.910794 31262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.911037 31247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:13.911140 31247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "1e9a8f78ee0c4ff48534ea2354523861"
format_stamp: "Formatted at 2026-06-20 08:13:13 on dist-test-slave-z0sp"
server_key: "7655ba822544c31cafe2f0fb5798118d"
server_key_iv: "76bcefcb536ffe39073ece712d4f8c2b"
server_key_version: "encryptionkey@0"
I20260620 08:13:13.911228 31247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:13.926829 31247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:13.929078 31247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:13.929252 31247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:13.929778 31247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:13.930341 31247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:13.930426 31247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.930462 31247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:13.930492 31247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:13.936425 31247 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:46317
I20260620 08:13:13.936475 31375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:46317 every 8 connection(s)
I20260620 08:13:13.936823 31247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
Jun 20 08:13:13 dist-test-slave-z0sp krb5kdc[30872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943193, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:13.943905 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31247
I20260620 08:13:13.944154 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:13.944561 12819 external_mini_cluster.cc:1428] Setting key 5c7f90a80f6ee93685c8dad17db23ba7
I20260620 08:13:13.947091 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.937640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:46359 (local address 127.12.132.254:45797)
0620 08:13:13.937720 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.937721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.938005 (+ 284us) server_negotiation.cc:407] Connection header received
0620 08:13:13.938418 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.938419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.938429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.938482 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:13.939417 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.940078 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.941271 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.941510 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.941571 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.943917 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:13.943919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:13.943920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:13.943931 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:13.945664 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.946110 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.946112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.946112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.946176 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.946436 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.946437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.946438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.946561 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:13.946637 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.946920 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.947000 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 08:13:13.947268 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.937539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:45797 (local address 127.12.132.195:46359)
0620 08:13:13.937785 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.937824 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.938281 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.938561 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.938565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.939101 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:13.939287 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.939291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.940269 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.940270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.941166 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.941169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.941599 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.941600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.941697 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.942188 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:13.942195 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:13.943788 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:13.945773 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.945781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.945782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.946012 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.946249 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:13.946249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:13.946250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:13.946363 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:13.946638 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:13.946645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:13.946777 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.946897 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.947049 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":111,"threads_started":1}
I20260620 08:13:13.948118 31376 heartbeater.cc:347] Connected to a master server at 127.12.132.254:45797
I20260620 08:13:13.948269 31376 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:13.948463 31376 heartbeater.cc:511] Master 127.12.132.254:45797 requested a full tablet report, sending...
I20260620 08:13:13.949040 30917 ts_manager.cc:194] Registered new tserver with Master: 1e9a8f78ee0c4ff48534ea2354523861 (127.12.132.195:46317)
I20260620 08:13:13.949597 30917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:46359
I20260620 08:13:13.956331 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:13.964282 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.957728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58212 (local address 127.12.132.254:45797)
0620 08:13:13.957815 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.957817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.957909 (+ 92us) server_negotiation.cc:407] Connection header received
0620 08:13:13.958000 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.958001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.958012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.958070 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:13.958615 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.959099 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.959804 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.960001 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.960088 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.961488 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:13.961490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:13.961490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:13.961497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:13.963013 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.963422 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.963424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.963424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.963478 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:13.963719 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:13.963720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:13.963720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:13.963860 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:13.963982 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.964125 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.964186 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 08:13:13.965974 30917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:13.967499 30917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:13.974824 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.12.132.195:46317)
0620 08:13:13.970853 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.970858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.970907 (+ 49us) server_negotiation.cc:407] Connection header received
0620 08:13:13.971276 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.971279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.971294 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.971368 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:13.971513 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972577 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.973618 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.974341 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.974427 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.974486 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.974701 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.974742 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260620 08:13:13.974854 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:34987 (local address 127.0.0.1:46416)
0620 08:13:13.970529 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.970539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.970594 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.971177 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.971178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.971180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:13.971217 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.971218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972170 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.972171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.973617 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.973620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.974483 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.974484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.974575 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.974592 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.974675 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.974747 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":7,"thread_start_us":69,"threads_started":1}
I20260620 08:13:13.975055 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46416 (local address 127.12.132.194:34987)
0620 08:13:13.971037 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.971040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.971047 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:13:13.971078 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.971079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.971089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.971179 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:13.971302 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972063 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.973716 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.974391 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.974459 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.974502 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.974939 (+ 437us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.974979 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":84,"threads_started":1}
I20260620 08:13:13.974855 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:46317 (local address 127.0.0.1:37348)
0620 08:13:13.970847 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.970858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.970915 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.971399 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.971401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.971403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:13.971433 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.971434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972715 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.972716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.973524 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.973527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.974481 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.974483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.974575 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.974592 (+ 17us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.974675 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.974746 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":25,"thread_start_us":37,"threads_started":1}
I20260620 08:13:13.975602 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40639 (local address 127.0.0.1:53390)
0620 08:13:13.970463 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:13.970503 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:13.970754 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:13.971035 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:13.971038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:13.971039 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:13.971094 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.971095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972456 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.972457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.973617 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:13.973620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.975370 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.975371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:13.975435 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.975438 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:13.975495 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 08:13:13.975533 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":25,"thread_start_us":91,"threads_started":1}
I20260620 08:13:13.975618 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:13.970380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.12.132.193:40639)
0620 08:13:13.970645 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:13.970650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:13.970662 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:13.970862 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:13.970865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:13.970881 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:13.970949 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:13.971176 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.972342 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.974536 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:13.975299 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:13.975372 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:13.975419 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:13.975507 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 08:13:13.975543 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":76,"threads_started":1}
I20260620 08:13:13.975775 31176 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3c520f53234704b8b4c75b7331992e (DEFAULT_TABLE table=test-table [id=f2153d7f51d646b4b419f6dae446017b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:13.976146 31176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3c520f53234704b8b4c75b7331992e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:13.976138 31310 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3c520f53234704b8b4c75b7331992e (DEFAULT_TABLE table=test-table [id=f2153d7f51d646b4b419f6dae446017b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:13.976449 31310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3c520f53234704b8b4c75b7331992e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:13.976362 31042 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3c520f53234704b8b4c75b7331992e (DEFAULT_TABLE table=test-table [id=f2153d7f51d646b4b419f6dae446017b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:13.976681 31042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3c520f53234704b8b4c75b7331992e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:13.978988 31390 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Bootstrap starting.
I20260620 08:13:13.979604 31391 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Bootstrap starting.
I20260620 08:13:13.979833 31390 tablet_bootstrap.cc:654] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:13.980259 31391 tablet_bootstrap.cc:654] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:13.980451 31390 log.cc:824] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:13.980656 31392 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Bootstrap starting.
I20260620 08:13:13.980856 31391 log.cc:824] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:13.981218 31390 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: No bootstrap required, opened a new log
I20260620 08:13:13.981276 31392 tablet_bootstrap.cc:654] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:13.981324 31390 ts_tablet_manager.cc:1397] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:13.981703 31391 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: No bootstrap required, opened a new log
I20260620 08:13:13.981774 31391 ts_tablet_manager.cc:1397] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 08:13:13.981868 31392 log.cc:824] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:13.982532 31392 tablet_bootstrap.cc:492] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: No bootstrap required, opened a new log
I20260620 08:13:13.982625 31392 ts_tablet_manager.cc:1397] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:13.983296 31390 raft_consensus.cc:348] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.983480 31390 raft_consensus.cc:374] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:13.983517 31390 raft_consensus.cc:729] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56fdfa60a4649438dfde063fecb46a2, State: Initialized, Role: FOLLOWER
I20260620 08:13:13.983636 31390 consensus_queue.cc:260] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.983670 31391 raft_consensus.cc:348] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.983839 31391 raft_consensus.cc:374] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:13.983886 31391 raft_consensus.cc:729] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e9a8f78ee0c4ff48534ea2354523861, State: Initialized, Role: FOLLOWER
I20260620 08:13:13.983923 31390 ts_tablet_manager.cc:1428] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:13:13.984025 31391 consensus_queue.cc:260] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.984063 31242 heartbeater.cc:503] Master 127.12.132.254:45797 was elected leader, sending a full tablet report...
I20260620 08:13:13.984277 31391 ts_tablet_manager.cc:1428] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 08:13:13.984433 31376 heartbeater.cc:503] Master 127.12.132.254:45797 was elected leader, sending a full tablet report...
I20260620 08:13:13.984210 31392 raft_consensus.cc:348] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.984508 31392 raft_consensus.cc:374] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:13.984550 31392 raft_consensus.cc:729] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d761ef7a75144ab18c23921f9bd25d79, State: Initialized, Role: FOLLOWER
I20260620 08:13:13.984652 31392 consensus_queue.cc:260] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:13.984933 31392 ts_tablet_manager.cc:1428] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 08:13:13.984948 31108 heartbeater.cc:503] Master 127.12.132.254:45797 was elected leader, sending a full tablet report...
W20260620 08:13:14.021481 31243 tablet.cc:2369] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:13:14.118968 31109 tablet.cc:2369] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:13:14.187511 31377 tablet.cc:2369] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:14.247143 31397 raft_consensus.cc:482] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:14.247332 31397 raft_consensus.cc:504] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:14.247786 31397 leader_election.cc:288] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d761ef7a75144ab18c23921f9bd25d79 (127.12.132.193:40639), c56fdfa60a4649438dfde063fecb46a2 (127.12.132.194:34987)
I20260620 08:13:14.251015 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.248150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60631 (local address 127.12.132.194:34987)
0620 08:13:14.248196 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.248197 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.248276 (+ 79us) server_negotiation.cc:407] Connection header received
0620 08:13:14.248310 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.248311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.248320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.248372 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:14.248584 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.249264 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250286 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.250648 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250725 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.250758 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.250902 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.250949 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:13:14.251039 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.247994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:34987 (local address 127.12.132.195:60631)
0620 08:13:14.248198 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.248208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.248265 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.248390 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.248391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.248392 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:14.248484 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.248486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.249349 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.249350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.250205 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.250207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.250747 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.250827 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.250831 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.250889 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.250940 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":16,"thread_start_us":101,"threads_started":1}
I20260620 08:13:14.251077 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.248055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:33673 (local address 127.12.132.193:40639)
0620 08:13:14.248126 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.248128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.248138 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:14.248171 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.248172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.248183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.248257 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:14.248463 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.249106 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250148 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.250662 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250769 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.250778 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.250966 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.251004 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:13:14.251359 31196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3c520f53234704b8b4c75b7331992e" candidate_uuid: "1e9a8f78ee0c4ff48534ea2354523861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56fdfa60a4649438dfde063fecb46a2" is_pre_election: true
I20260620 08:13:14.251423 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.247972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40639 (local address 127.12.132.195:33673)
0620 08:13:14.248029 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.248041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.248094 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.248284 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.248285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.248287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:14.248377 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.248378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.249181 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.249182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.250072 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.250074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.250785 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.250786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.250881 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.250884 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.250942 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.250978 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:13:14.251506 31196 raft_consensus.cc:2454] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e9a8f78ee0c4ff48534ea2354523861 in term 0.
I20260620 08:13:14.251724 31266 leader_election.cc:302] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e9a8f78ee0c4ff48534ea2354523861, c56fdfa60a4649438dfde063fecb46a2; no voters:
I20260620 08:13:14.251801 31062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3c520f53234704b8b4c75b7331992e" candidate_uuid: "1e9a8f78ee0c4ff48534ea2354523861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d761ef7a75144ab18c23921f9bd25d79" is_pre_election: true
I20260620 08:13:14.251888 31397 raft_consensus.cc:2781] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:14.251926 31062 raft_consensus.cc:2454] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e9a8f78ee0c4ff48534ea2354523861 in term 0.
I20260620 08:13:14.251962 31397 raft_consensus.cc:482] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:14.251991 31397 raft_consensus.cc:3037] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:14.252790 31397 raft_consensus.cc:504] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:14.252976 31397 leader_election.cc:288] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [CANDIDATE]: Term 1 election: Requested vote from peers d761ef7a75144ab18c23921f9bd25d79 (127.12.132.193:40639), c56fdfa60a4649438dfde063fecb46a2 (127.12.132.194:34987)
I20260620 08:13:14.253126 31196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3c520f53234704b8b4c75b7331992e" candidate_uuid: "1e9a8f78ee0c4ff48534ea2354523861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c56fdfa60a4649438dfde063fecb46a2"
I20260620 08:13:14.253144 31062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3c520f53234704b8b4c75b7331992e" candidate_uuid: "1e9a8f78ee0c4ff48534ea2354523861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d761ef7a75144ab18c23921f9bd25d79"
I20260620 08:13:14.253226 31196 raft_consensus.cc:3037] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:14.253223 31062 raft_consensus.cc:3037] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:14.254097 31062 raft_consensus.cc:2454] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e9a8f78ee0c4ff48534ea2354523861 in term 1.
I20260620 08:13:14.254124 31196 raft_consensus.cc:2454] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e9a8f78ee0c4ff48534ea2354523861 in term 1.
I20260620 08:13:14.254292 31266 leader_election.cc:302] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e9a8f78ee0c4ff48534ea2354523861, c56fdfa60a4649438dfde063fecb46a2; no voters:
I20260620 08:13:14.254376 31397 raft_consensus.cc:2781] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:14.254521 31397 raft_consensus.cc:686] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 1 LEADER]: Becoming Leader. State: Replica: 1e9a8f78ee0c4ff48534ea2354523861, State: Running, Role: LEADER
I20260620 08:13:14.254626 31397 consensus_queue.cc:237] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } }
I20260620 08:13:14.255419 30915 catalog_manager.cc:5495] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e9a8f78ee0c4ff48534ea2354523861 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "1e9a8f78ee0c4ff48534ea2354523861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e9a8f78ee0c4ff48534ea2354523861" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 46317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 } health_report { overall_health: UNKNOWN } } }
I20260620 08:13:14.329312 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.326813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.12.132.195:46317)
0620 08:13:14.326913 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.326914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.326927 (+ 13us) server_negotiation.cc:407] Connection header received
0620 08:13:14.326966 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.326967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.326984 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.327049 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:14.327296 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.327761 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.328612 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.328808 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.328876 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.329019 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:14.329105 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:14.329177 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.329228 (+ 51us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.329237 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 08:13:14.331152 31062 raft_consensus.cc:1261] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 1 FOLLOWER]: Refusing update from remote peer 1e9a8f78ee0c4ff48534ea2354523861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:14.331171 31196 raft_consensus.cc:1261] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 1 FOLLOWER]: Refusing update from remote peer 1e9a8f78ee0c4ff48534ea2354523861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:14.331537 31400 consensus_queue.cc:1035] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c56fdfa60a4649438dfde063fecb46a2" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:14.331686 31397 consensus_queue.cc:1035] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d761ef7a75144ab18c23921f9bd25d79" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:14.334427 31403 mvcc.cc:205] Tried to move back new op lower bound from 7298839323977662464 to 7298839323667582976. Current Snapshot: MvccSnapshot[applied={T|T < 7298839323977662464 or (T in {7298839323977662464})}]
I20260620 08:13:14.334537 31404 mvcc.cc:205] Tried to move back new op lower bound from 7298839323977662464 to 7298839323667582976. Current Snapshot: MvccSnapshot[applied={T|T < 7298839323977662464}]
I20260620 08:13:14.334942 31405 mvcc.cc:205] Tried to move back new op lower bound from 7298839323977662464 to 7298839323667582976. Current Snapshot: MvccSnapshot[applied={T|T < 7298839323977662464 or (T in {7298839323977662464})}]
I20260620 08:13:14.336467 30917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:14.336570 30917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:14.337361 30917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57301ac0532a495f99f96f79eebdc579: Sending DeleteTablet for 3 replicas of tablet 2b3c520f53234704b8b4c75b7331992e
I20260620 08:13:14.337715 31310 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3c520f53234704b8b4c75b7331992e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:14 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:37348
I20260620 08:13:14.337673 31042 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3c520f53234704b8b4c75b7331992e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:14 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:53390
I20260620 08:13:14.337842 31176 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3c520f53234704b8b4c75b7331992e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:14 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:46416
I20260620 08:13:14.338209 31414 tablet_replica.cc:330] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: stopping tablet replica
I20260620 08:13:14.338233 31413 tablet_replica.cc:330] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79: stopping tablet replica
I20260620 08:13:14.338297 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30979
I20260620 08:13:14.338346 31414 raft_consensus.cc:2229] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:14.338404 31413 raft_consensus.cc:2229] T 2b3c520f53234704b8b4c75b7331992e P d761ef7a75144ab18c23921f9bd25d79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:14.338441 31414 raft_consensus.cc:2258] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:14.339341 31414 ts_tablet_manager.cc:1905] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:14.341014 31414 ts_tablet_manager.cc:1918] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:14.341077 31414 log.cc:1196] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/2b3c520f53234704b8b4c75b7331992e
I20260620 08:13:14.341338 31414 ts_tablet_manager.cc:1939] T 2b3c520f53234704b8b4c75b7331992e P c56fdfa60a4649438dfde063fecb46a2: Deleting consensus metadata
I20260620 08:13:14.341908 30904 catalog_manager.cc:4841] TS c56fdfa60a4649438dfde063fecb46a2 (127.12.132.194:34987): tablet 2b3c520f53234704b8b4c75b7331992e (table test-table [id=f2153d7f51d646b4b419f6dae446017b]) successfully deleted
W20260620 08:13:14.344898 30903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:40639 (error 108)
I20260620 08:13:14.345445 31415 tablet_replica.cc:330] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: stopping tablet replica
I20260620 08:13:14.345496 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.345320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:40639 (local address 127.0.0.1:53396)
0620 08:13:14.345413 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.345460 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 08:13:14.345562 31415 raft_consensus.cc:2229] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:14.345655 31415 raft_consensus.cc:2258] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:14.345888 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31113
W20260620 08:13:14.345865 30903 catalog_manager.cc:4584] TS d761ef7a75144ab18c23921f9bd25d79 (127.12.132.193:40639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b3c520f53234704b8b4c75b7331992e: Network error: Client connection negotiation failed: client connection to 127.12.132.193:40639: connect: Connection refused (error 111)
I20260620 08:13:14.351956 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31247
I20260620 08:13:14.352090 31415 ts_tablet_manager.cc:1905] T 2b3c520f53234704b8b4c75b7331992e P 1e9a8f78ee0c4ff48534ea2354523861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:14.356961 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.356806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:46317 (local address 127.0.0.1:37372)
0620 08:13:14.356891 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.356929 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.195:46317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 08:13:14.357044 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 30885
2026-06-20T08:13:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2205 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T08:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:14Z Disabled control of system clock
I20260620 08:13:14.385625 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38955
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:14.467854 31420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:14.468062 31420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:14.468082 31420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:14.469238 31420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:14.469286 31420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:14.469300 31420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:14.469312 31420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:14.469324 31420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:14.470570 31420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:14.470784 31420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:14.471006 31420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:14.475955 31425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.475907 31428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.475979 31426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:14.476164 31420 server_base.cc:1034] running on GCE node
I20260620 08:13:14.476645 31420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:14.477093 31420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:14.478279 31420 hybrid_clock.cc:648] HybridClock initialized: now 1781943194478263 us; error 29 us; skew 500 ppm
I20260620 08:13:14.479645 31420 webserver.cc:460] Webserver started at http://127.12.132.254:46647/ using document root <none> and password file <none>
I20260620 08:13:14.479933 31420 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:14.480005 31420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:14.480136 31420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:14.481259 31420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "dee2fbb470234410ae15029f7ee4205f"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "1d9fd1b10524fccd74ca7f9e47a3ef0f"
server_key_iv: "365ffcab0b5fee751c0eaa6217157421"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.481630 31420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "dee2fbb470234410ae15029f7ee4205f"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "1d9fd1b10524fccd74ca7f9e47a3ef0f"
server_key_iv: "365ffcab0b5fee751c0eaa6217157421"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.483085 31420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:14.483983 31434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.484297 31420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:14.484375 31420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "dee2fbb470234410ae15029f7ee4205f"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "1d9fd1b10524fccd74ca7f9e47a3ef0f"
server_key_iv: "365ffcab0b5fee751c0eaa6217157421"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.484469 31420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:14.496690 31420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:14.497104 31420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:14.497295 31420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:14.502763 31420 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38955
I20260620 08:13:14.502869 31486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38955 every 8 connection(s)
I20260620 08:13:14.503240 31420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:14.504271 31487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:14.506541 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: Bootstrap starting.
I20260620 08:13:14.507051 31487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:14.507292 31487 log.cc:824] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:14.508009 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: No bootstrap required, opened a new log
I20260620 08:13:14.509649 31487 raft_consensus.cc:348] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } }
I20260620 08:13:14.509781 31487 raft_consensus.cc:374] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:14.509804 31487 raft_consensus.cc:729] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee2fbb470234410ae15029f7ee4205f, State: Initialized, Role: FOLLOWER
I20260620 08:13:14.509945 31487 consensus_queue.cc:260] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } }
I20260620 08:13:14.510028 31487 raft_consensus.cc:388] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:14.510051 31487 raft_consensus.cc:482] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:14.510079 31487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:14.510452 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31420
I20260620 08:13:14.510591 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:14.510535 31487 raft_consensus.cc:504] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } }
I20260620 08:13:14.510711 31487 leader_election.cc:302] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dee2fbb470234410ae15029f7ee4205f; no voters:
I20260620 08:13:14.510844 12819 external_mini_cluster.cc:1428] Setting key 37b5fb9b2f0ed6e75ee055b46d89c525
I20260620 08:13:14.510910 31487 leader_election.cc:288] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:14.510944 31490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:14.511116 31490 raft_consensus.cc:686] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [term 1 LEADER]: Becoming Leader. State: Replica: dee2fbb470234410ae15029f7ee4205f, State: Running, Role: LEADER
I20260620 08:13:14.511176 31487 sys_catalog.cc:564] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:14.511257 31490 consensus_queue.cc:237] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } }
I20260620 08:13:14.511572 31490 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dee2fbb470234410ae15029f7ee4205f. Latest consensus state: current_term: 1 leader_uuid: "dee2fbb470234410ae15029f7ee4205f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } } }
I20260620 08:13:14.511686 31492 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dee2fbb470234410ae15029f7ee4205f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee2fbb470234410ae15029f7ee4205f" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38955 } } }
I20260620 08:13:14.511826 31492 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:14.512089 31501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:14.511731 31490 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:14.512656 31501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:14.514331 31501 catalog_manager.cc:1347] Generated new cluster ID: 18dc4eef448c4997907c30307986434a
I20260620 08:13:14.514385 31501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:14.517004 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.511522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.12.132.254:38955)
0620 08:13:14.511952 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.511961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.511995 (+ 34us) server_negotiation.cc:407] Connection header received
0620 08:13:14.512109 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.512115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.512184 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.512329 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:14.512701 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.513818 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.514850 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.515439 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.515576 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.516266 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:14.516270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:14.516270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:14.516478 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:14.516554 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:14.516570 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:14.516639 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.516706 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.516789 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":78,"threads_started":1}
I20260620 08:13:14.520923 31501 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:14.521409 31501 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:14.532397 31501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: Generated new TSK 0
I20260620 08:13:14.532622 31501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:14.539271 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--builtin_ntp_servers=127.12.132.212:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:14.622592 31512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:14.622803 31512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:14.622848 31512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:14.624141 31512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:14.624181 31512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:14.624207 31512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:14.625594 31512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:14.625857 31512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:14.626178 31512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:14.632690 31518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.632700 31520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.632705 31517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:14.633019 31512 server_base.cc:1034] running on GCE node
I20260620 08:13:14.633224 31512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:14.633464 31512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:14.634676 31512 hybrid_clock.cc:648] HybridClock initialized: now 1781943194634653 us; error 36 us; skew 500 ppm
I20260620 08:13:14.636029 31512 webserver.cc:460] Webserver started at http://127.12.132.193:42099/ using document root <none> and password file <none>
I20260620 08:13:14.636276 31512 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:14.636345 31512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:14.636482 31512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:14.637625 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "fb1f4479fe92449f98a80208704ccb29"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "8b604d4ab77fe624bf8173ea6863c794"
server_key_iv: "d08298a3b20e8aa1952e18ea4d41990e"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.637990 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "fb1f4479fe92449f98a80208704ccb29"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "8b604d4ab77fe624bf8173ea6863c794"
server_key_iv: "d08298a3b20e8aa1952e18ea4d41990e"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.639498 31512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:14.640309 31526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.640681 31512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:14.640862 31512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "fb1f4479fe92449f98a80208704ccb29"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "8b604d4ab77fe624bf8173ea6863c794"
server_key_iv: "d08298a3b20e8aa1952e18ea4d41990e"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.641033 31512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:14.659345 31512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:14.659699 31512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:14.659862 31512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:14.660125 31512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:14.660475 31512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:14.660562 31512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.660601 31512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:14.660645 31512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.666617 31512 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:36005
I20260620 08:13:14.666710 31639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:36005 every 8 connection(s)
I20260620 08:13:14.667097 31512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:13:14.672772 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.668088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52463 (local address 127.12.132.254:38955)
0620 08:13:14.668228 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.668231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.668361 (+ 130us) server_negotiation.cc:407] Connection header received
0620 08:13:14.668912 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.668913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.668928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.668980 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:14.669639 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.670138 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.671265 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.671467 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.671550 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.672449 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:14.672450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:14.672450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:14.672455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:14.672470 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:14.672480 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:14.672566 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.672662 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.672692 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260620 08:13:14.673015 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.667988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38955 (local address 127.12.132.193:52463)
0620 08:13:14.668210 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.668252 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.668799 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.669046 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.669050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.669063 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:14.669542 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.669546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.670243 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.670244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.671152 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.671155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.671558 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.671559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.671699 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.672307 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:14.672308 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:14.672335 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:14.672356 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:14.672567 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:14.672574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:14.672576 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.672664 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.672776 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260620 08:13:14.673543 31640 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38955
I20260620 08:13:14.673628 31640 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:14.673858 31640 heartbeater.cc:511] Master 127.12.132.254:38955 requested a full tablet report, sending...
I20260620 08:13:14.674379 31451 ts_manager.cc:194] Registered new tserver with Master: fb1f4479fe92449f98a80208704ccb29 (127.12.132.193:36005)
I20260620 08:13:14.674749 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31512
I20260620 08:13:14.674866 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:14.675067 12819 external_mini_cluster.cc:1428] Setting key a14a67609d55cc0e95ab59c04249edbe
I20260620 08:13:14.675077 31451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.193:52463
I20260620 08:13:14.676080 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--builtin_ntp_servers=127.12.132.212:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:14.759907 31643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:14.760133 31643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:14.760167 31643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:14.761345 31643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:14.761405 31643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:14.761448 31643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:14.763931 31643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:14.764285 31643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:14.764674 31643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:14.769677 31649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.769752 31648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:14.769778 31643 server_base.cc:1034] running on GCE node
W20260620 08:13:14.769680 31651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:14.770162 31643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:14.770411 31643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:14.771597 31643 hybrid_clock.cc:648] HybridClock initialized: now 1781943194771567 us; error 42 us; skew 500 ppm
I20260620 08:13:14.773018 31643 webserver.cc:460] Webserver started at http://127.12.132.194:39539/ using document root <none> and password file <none>
I20260620 08:13:14.773290 31643 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:14.773365 31643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:14.773520 31643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:14.774554 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "880f394cfcfc4ef69952f1f6b28bfb84"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "996ed46e7173ae0ba61b107fb9559bb4"
server_key_iv: "1858142d611fe144a05387217190a9fd"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.775111 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "880f394cfcfc4ef69952f1f6b28bfb84"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "996ed46e7173ae0ba61b107fb9559bb4"
server_key_iv: "1858142d611fe144a05387217190a9fd"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.776798 31643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:14.777594 31657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.777907 31643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:14.778008 31643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "880f394cfcfc4ef69952f1f6b28bfb84"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "996ed46e7173ae0ba61b107fb9559bb4"
server_key_iv: "1858142d611fe144a05387217190a9fd"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.778093 31643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:14.807482 31643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:14.807828 31643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:14.807986 31643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:14.808230 31643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:14.808609 31643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:14.808670 31643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.808713 31643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:14.808740 31643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.815169 31643 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:33241
I20260620 08:13:14.815214 31770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:33241 every 8 connection(s)
I20260620 08:13:14.815557 31643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:13:14.821229 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.816688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:33793 (local address 127.12.132.254:38955)
0620 08:13:14.816835 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.816837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.817015 (+ 178us) server_negotiation.cc:407] Connection header received
0620 08:13:14.817541 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.817543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.817555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.817619 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:14.818299 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.818790 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.819782 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.819975 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.820061 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.820937 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:14.820938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:14.820939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:14.820944 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:14.820957 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:14.820962 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:14.821016 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.821112 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.821139 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260620 08:13:14.821439 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.816607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38955 (local address 127.12.132.194:33793)
0620 08:13:14.816890 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.816924 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.817434 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.817691 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.817695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.817702 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:14.818192 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.818196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.818885 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.818887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.819662 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.819663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.820073 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.820074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.820200 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.820768 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:14.820768 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:14.820796 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:14.820817 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:14.821022 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:14.821027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:14.821030 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.821114 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.821210 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260620 08:13:14.821511 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31643
I20260620 08:13:14.821611 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:14.821836 12819 external_mini_cluster.cc:1428] Setting key b344fe445b5984218c313a55937fb19e
I20260620 08:13:14.822043 31771 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38955
I20260620 08:13:14.822147 31771 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:14.822335 31771 heartbeater.cc:511] Master 127.12.132.254:38955 requested a full tablet report, sending...
I20260620 08:13:14.822656 31451 ts_manager.cc:194] Registered new tserver with Master: 880f394cfcfc4ef69952f1f6b28bfb84 (127.12.132.194:33241)
I20260620 08:13:14.822814 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--builtin_ntp_servers=127.12.132.212:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 08:13:14.822974 31451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.194:33793
W20260620 08:13:14.907217 31774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:14.907418 31774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:14.907441 31774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:14.908643 31774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:14.908689 31774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:14.908716 31774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:14.910364 31774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:38955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:14.910578 31774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:14.910915 31774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:14.915987 31780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.915987 31782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:14.916057 31779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:14.916088 31774 server_base.cc:1034] running on GCE node
I20260620 08:13:14.916390 31774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:14.916706 31774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:14.917914 31774 hybrid_clock.cc:648] HybridClock initialized: now 1781943194917890 us; error 33 us; skew 500 ppm
I20260620 08:13:14.919235 31774 webserver.cc:460] Webserver started at http://127.12.132.195:44423/ using document root <none> and password file <none>
I20260620 08:13:14.919481 31774 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:14.919552 31774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:14.919687 31774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:14.920800 31774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "f4657efa9edf4e46a045d0b6b9292544"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "39e0c115c10f4b78a61342cbdce12d6f"
server_key_iv: "e74a573bd8a8f419ee285b82a08b1097"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.921162 31774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "f4657efa9edf4e46a045d0b6b9292544"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "39e0c115c10f4b78a61342cbdce12d6f"
server_key_iv: "e74a573bd8a8f419ee285b82a08b1097"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.922467 31774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:14.923593 31788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.924027 31774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:14.924189 31774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "f4657efa9edf4e46a045d0b6b9292544"
format_stamp: "Formatted at 2026-06-20 08:13:14 on dist-test-slave-z0sp"
server_key: "39e0c115c10f4b78a61342cbdce12d6f"
server_key_iv: "e74a573bd8a8f419ee285b82a08b1097"
server_key_version: "encryptionkey@0"
I20260620 08:13:14.924311 31774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:14.949986 31774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:14.950315 31774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:14.950469 31774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:14.950711 31774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:14.951066 31774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:14.951119 31774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.951155 31774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:14.951181 31774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:14.957618 31774 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:42715
I20260620 08:13:14.957669 31901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:42715 every 8 connection(s)
I20260620 08:13:14.957994 31774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:13:14.958484 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31774
I20260620 08:13:14.958606 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:14.958858 12819 external_mini_cluster.cc:1428] Setting key 13caeb3feb2561528c3968e1f6cb0745
I20260620 08:13:14.964097 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.958688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:56017 (local address 127.12.132.254:38955)
0620 08:13:14.958769 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.958772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.959113 (+ 341us) server_negotiation.cc:407] Connection header received
0620 08:13:14.959747 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.959747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.959762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.959823 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:14.960488 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.961060 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.962082 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.962348 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.962463 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.963729 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:14.963730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:14.963731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:14.963737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:14.963754 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:14.963759 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:14.963820 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.964005 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.964038 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 08:13:14.964243 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.958683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:38955 (local address 127.12.132.195:56017)
0620 08:13:14.958855 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.958897 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.959598 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.959890 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.959894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.959904 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:14.960395 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.960398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.961154 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.961155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.961931 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.961933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.962555 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.962557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.962742 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.963539 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:14.963540 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:14.963578 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:14.963609 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:14.963864 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:14.963871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:14.963874 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.963959 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.964052 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260620 08:13:14.964731 31902 heartbeater.cc:347] Connected to a master server at 127.12.132.254:38955
I20260620 08:13:14.964860 31902 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:14.965039 31902 heartbeater.cc:511] Master 127.12.132.254:38955 requested a full tablet report, sending...
I20260620 08:13:14.965471 31451 ts_manager.cc:194] Registered new tserver with Master: f4657efa9edf4e46a045d0b6b9292544 (127.12.132.195:42715)
I20260620 08:13:14.965862 31451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.195:56017
I20260620 08:13:14.970556 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:14.975224 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.972007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.12.132.254:38955)
0620 08:13:14.972068 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.972069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.972190 (+ 121us) server_negotiation.cc:407] Connection header received
0620 08:13:14.972213 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.972213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.972225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.972279 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:14.972426 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.972960 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.973704 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.973881 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.973928 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.974741 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:14.974744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:14.974744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:14.974751 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:14.974775 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:14.974782 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:14.974932 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.975117 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.975147 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:13:14.977135 31451 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:14.978694 31451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:14.985466 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.12.132.195:42715)
0620 08:13:14.982029 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.982034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.982044 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:14.982080 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.982082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.982098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.982225 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:14.982395 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.983228 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.984470 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.985083 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.985162 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.985242 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.985351 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.985385 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260620 08:13:14.985476 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42715 (local address 127.0.0.1:44254)
0620 08:13:14.981625 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.981637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.981682 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.982238 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.982241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.982243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:14.982308 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.982311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.983340 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.983341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.984398 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.984400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.985171 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.985172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.985247 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.985261 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.985338 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.985394 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":182,"threads_started":1}
I20260620 08:13:14.986032 31910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:36005 (local address 127.0.0.1:48922)
0620 08:13:14.981434 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.981473 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.981624 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.982271 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.982272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.982276 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:14.982352 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.982356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.983912 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.983913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.984823 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.984825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.985814 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.985815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.985875 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.985880 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.985934 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.985971 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":67,"threads_started":1}
I20260620 08:13:14.986073 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.12.132.193:36005)
0620 08:13:14.982043 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.982051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.982066 (+ 15us) server_negotiation.cc:407] Connection header received
0620 08:13:14.982121 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.982123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.982143 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.982466 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:14.982534 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.983789 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.984923 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.985711 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.985804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.985893 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.985949 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.985995 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":95,"threads_started":1}
I20260620 08:13:14.986428 31836 tablet_service.cc:1467] Processing CreateTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e (DEFAULT_TABLE table=test-table [id=bc3dcacdad1342419a49aeb2c472728f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:14.986830 31836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44a7ea8fcf7b4842acdab4d81186921e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:14.987217 31915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36388 (local address 127.12.132.194:33241)
0620 08:13:14.982305 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:14.982309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:14.982320 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:14.982365 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:14.982367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:14.982383 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:14.982465 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:14.982582 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.983787 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.985947 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.986896 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.986976 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.987030 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:14.987129 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 08:13:14.987159 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":102,"threads_started":1}
I20260620 08:13:14.987246 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:14.981643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:33241 (local address 127.0.0.1:36388)
0620 08:13:14.981782 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:14.981792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:14.982083 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:14.982472 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:14.982472 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:14.982475 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:14.982514 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.982515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.984904 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.984905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.985838 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:14.985840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:14.986986 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:14.986987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:14.987049 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:14.987054 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:13:14.987121 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:13:14.987163 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260620 08:13:14.987852 31705 tablet_service.cc:1467] Processing CreateTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e (DEFAULT_TABLE table=test-table [id=bc3dcacdad1342419a49aeb2c472728f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:14.988137 31705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44a7ea8fcf7b4842acdab4d81186921e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:14.988947 31574 tablet_service.cc:1467] Processing CreateTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e (DEFAULT_TABLE table=test-table [id=bc3dcacdad1342419a49aeb2c472728f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:14.989264 31574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44a7ea8fcf7b4842acdab4d81186921e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:14.989689 31916 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Bootstrap starting.
I20260620 08:13:14.990272 31916 tablet_bootstrap.cc:654] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:14.990558 31916 log.cc:824] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:14.991209 31916 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: No bootstrap required, opened a new log
I20260620 08:13:14.991293 31916 ts_tablet_manager.cc:1397] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:14.991479 31918 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Bootstrap starting.
I20260620 08:13:14.992148 31918 tablet_bootstrap.cc:654] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:14.992552 31919 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: Bootstrap starting.
I20260620 08:13:14.993192 31919 tablet_bootstrap.cc:654] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:14.993518 31918 log.cc:824] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:14.993544 31919 log.cc:824] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:14.994261 31918 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: No bootstrap required, opened a new log
I20260620 08:13:14.994316 31919 tablet_bootstrap.cc:492] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: No bootstrap required, opened a new log
I20260620 08:13:14.994346 31918 ts_tablet_manager.cc:1397] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 08:13:14.994385 31919 ts_tablet_manager.cc:1397] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 08:13:14.994534 31916 raft_consensus.cc:348] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.994695 31916 raft_consensus.cc:374] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:14.994745 31916 raft_consensus.cc:729] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4657efa9edf4e46a045d0b6b9292544, State: Initialized, Role: FOLLOWER
I20260620 08:13:14.994908 31916 consensus_queue.cc:260] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.995201 31916 ts_tablet_manager.cc:1428] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 08:13:14.995234 31902 heartbeater.cc:503] Master 127.12.132.254:38955 was elected leader, sending a full tablet report...
I20260620 08:13:14.996778 31918 raft_consensus.cc:348] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.996923 31918 raft_consensus.cc:374] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:14.996824 31919 raft_consensus.cc:348] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.996966 31918 raft_consensus.cc:729] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880f394cfcfc4ef69952f1f6b28bfb84, State: Initialized, Role: FOLLOWER
I20260620 08:13:14.996974 31919 raft_consensus.cc:374] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:14.997015 31919 raft_consensus.cc:729] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb1f4479fe92449f98a80208704ccb29, State: Initialized, Role: FOLLOWER
I20260620 08:13:14.997064 31918 consensus_queue.cc:260] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.997177 31919 consensus_queue.cc:260] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:14.997310 31918 ts_tablet_manager.cc:1428] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:13:14.997340 31771 heartbeater.cc:503] Master 127.12.132.254:38955 was elected leader, sending a full tablet report...
I20260620 08:13:14.997433 31919 ts_tablet_manager.cc:1428] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 08:13:14.997579 31640 heartbeater.cc:503] Master 127.12.132.254:38955 was elected leader, sending a full tablet report...
I20260620 08:13:15.004431 31924 raft_consensus.cc:482] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:15.004601 31924 raft_consensus.cc:504] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:15.004905 31924 leader_election.cc:288] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4657efa9edf4e46a045d0b6b9292544 (127.12.132.195:42715), 880f394cfcfc4ef69952f1f6b28bfb84 (127.12.132.194:33241)
I20260620 08:13:15.007679 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.005091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42715 (local address 127.12.132.193:52053)
0620 08:13:15.005145 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:15.005158 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:15.005201 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:15.005453 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:15.005454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:15.005455 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:15.005510 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:15.005511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.006141 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.006142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:15.006891 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:15.006892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.007418 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.007419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:15.007493 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:15.007502 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 08:13:15.007573 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 08:13:15.007612 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 08:13:15.007678 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.005209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:52053 (local address 127.12.132.195:42715)
0620 08:13:15.005283 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:15.005285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:15.005294 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:15.005319 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:15.005320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:15.005329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:15.005423 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:15.005580 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.006058 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.006977 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.007331 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.007409 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:15.007417 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:15.007581 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 08:13:15.007617 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 08:13:15.008018 31915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.005273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:38447 (local address 127.12.132.194:33241)
0620 08:13:15.005344 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:15.005346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:15.005472 (+ 126us) server_negotiation.cc:407] Connection header received
0620 08:13:15.005542 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:15.005543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:15.005553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:15.005592 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:15.005745 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.006208 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.007132 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.007504 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.007594 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:15.007601 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:15.007919 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 08:13:15.007958 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:13:15.007983 31856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44a7ea8fcf7b4842acdab4d81186921e" candidate_uuid: "fb1f4479fe92449f98a80208704ccb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4657efa9edf4e46a045d0b6b9292544" is_pre_election: true
I20260620 08:13:15.008037 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.005229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:33241 (local address 127.12.132.193:38447)
0620 08:13:15.005418 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:15.005429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:15.005485 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:15.005603 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:15.005604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:15.005605 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:15.005670 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:15.005672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.006303 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.006303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:15.007045 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:15.007046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.007588 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.007589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:15.007667 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:15.007670 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:15.007910 (+ 240us) client_negotiation.cc:240] Negotiation successful
0620 08:13:15.007952 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260620 08:13:15.008116 31856 raft_consensus.cc:2454] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb1f4479fe92449f98a80208704ccb29 in term 0.
I20260620 08:13:15.008421 31527 leader_election.cc:302] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f4657efa9edf4e46a045d0b6b9292544, fb1f4479fe92449f98a80208704ccb29; no voters:
I20260620 08:13:15.008559 31924 raft_consensus.cc:2781] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:15.008651 31924 raft_consensus.cc:482] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:15.008683 31924 raft_consensus.cc:3037] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:15.008831 31725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44a7ea8fcf7b4842acdab4d81186921e" candidate_uuid: "fb1f4479fe92449f98a80208704ccb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" is_pre_election: true
I20260620 08:13:15.008970 31725 raft_consensus.cc:2454] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb1f4479fe92449f98a80208704ccb29 in term 0.
I20260620 08:13:15.009514 31924 raft_consensus.cc:504] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:15.009645 31924 leader_election.cc:288] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [CANDIDATE]: Term 1 election: Requested vote from peers f4657efa9edf4e46a045d0b6b9292544 (127.12.132.195:42715), 880f394cfcfc4ef69952f1f6b28bfb84 (127.12.132.194:33241)
I20260620 08:13:15.009809 31725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44a7ea8fcf7b4842acdab4d81186921e" candidate_uuid: "fb1f4479fe92449f98a80208704ccb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880f394cfcfc4ef69952f1f6b28bfb84"
I20260620 08:13:15.009828 31856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44a7ea8fcf7b4842acdab4d81186921e" candidate_uuid: "fb1f4479fe92449f98a80208704ccb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4657efa9edf4e46a045d0b6b9292544"
I20260620 08:13:15.009898 31856 raft_consensus.cc:3037] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:15.009920 31725 raft_consensus.cc:3037] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:15.010581 31856 raft_consensus.cc:2454] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb1f4479fe92449f98a80208704ccb29 in term 1.
I20260620 08:13:15.010593 31725 raft_consensus.cc:2454] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb1f4479fe92449f98a80208704ccb29 in term 1.
I20260620 08:13:15.010747 31527 leader_election.cc:302] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f4657efa9edf4e46a045d0b6b9292544, fb1f4479fe92449f98a80208704ccb29; no voters:
I20260620 08:13:15.010854 31924 raft_consensus.cc:2781] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:15.010994 31924 raft_consensus.cc:686] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 1 LEADER]: Becoming Leader. State: Replica: fb1f4479fe92449f98a80208704ccb29, State: Running, Role: LEADER
I20260620 08:13:15.011150 31924 consensus_queue.cc:237] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } }
I20260620 08:13:15.012030 31451 catalog_manager.cc:5495] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb1f4479fe92449f98a80208704ccb29 (127.12.132.193). New cstate: current_term: 1 leader_uuid: "fb1f4479fe92449f98a80208704ccb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb1f4479fe92449f98a80208704ccb29" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 36005 } health_report { overall_health: HEALTHY } } }
I20260620 08:13:15.021603 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.019237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48934 (local address 127.12.132.193:36005)
0620 08:13:15.019315 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:15.019317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:15.019325 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:13:15.019354 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:15.019355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:15.019380 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:15.019435 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:15.019542 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.020006 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.020825 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:15.021039 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:15.021098 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:15.021259 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:15.021340 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:15.021419 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:15.021461 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 08:13:15.021469 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 08:13:15.023267 31725 raft_consensus.cc:1261] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 1 FOLLOWER]: Refusing update from remote peer fb1f4479fe92449f98a80208704ccb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:15.023283 31856 raft_consensus.cc:1261] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 1 FOLLOWER]: Refusing update from remote peer fb1f4479fe92449f98a80208704ccb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:15.023633 31926 consensus_queue.cc:1035] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880f394cfcfc4ef69952f1f6b28bfb84" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 33241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:15.023741 31924 consensus_queue.cc:1035] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4657efa9edf4e46a045d0b6b9292544" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:15.026774 31930 mvcc.cc:205] Tried to move back new op lower bound from 7298839326812839936 to 7298839326766366720. Current Snapshot: MvccSnapshot[applied={T|T < 7298839326812839936}]
I20260620 08:13:15.027168 31931 mvcc.cc:205] Tried to move back new op lower bound from 7298839326812839936 to 7298839326766366720. Current Snapshot: MvccSnapshot[applied={T|T < 7298839326812839936 or (T in {7298839326812839936})}]
I20260620 08:13:15.028893 31450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:15.028993 31450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:15.029803 31450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dee2fbb470234410ae15029f7ee4205f: Sending DeleteTablet for 3 replicas of tablet 44a7ea8fcf7b4842acdab4d81186921e
I20260620 08:13:15.030110 31836 tablet_service.cc:1514] Processing DeleteTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:15 UTC) from {username='slave'} at 127.0.0.1:44254
I20260620 08:13:15.030143 31574 tablet_service.cc:1514] Processing DeleteTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:15 UTC) from {username='slave'} at 127.0.0.1:48922
I20260620 08:13:15.030180 31705 tablet_service.cc:1514] Processing DeleteTablet for tablet 44a7ea8fcf7b4842acdab4d81186921e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:15 UTC) from {username='slave'} at 127.0.0.1:36388
I20260620 08:13:15.030362 31939 tablet_replica.cc:330] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29: stopping tablet replica
I20260620 08:13:15.030458 31939 raft_consensus.cc:2229] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:15.030560 31939 raft_consensus.cc:2258] T 44a7ea8fcf7b4842acdab4d81186921e P fb1f4479fe92449f98a80208704ccb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:15.030568 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31512
I20260620 08:13:15.031129 31940 tablet_replica.cc:330] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: stopping tablet replica
I20260620 08:13:15.031243 31940 raft_consensus.cc:2229] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:15.031338 31940 raft_consensus.cc:2258] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:15.031731 31940 ts_tablet_manager.cc:1905] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:15.033392 31940 ts_tablet_manager.cc:1918] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:15.033483 31940 log.cc:1196] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/44a7ea8fcf7b4842acdab4d81186921e
I20260620 08:13:15.033699 31940 ts_tablet_manager.cc:1939] T 44a7ea8fcf7b4842acdab4d81186921e P 880f394cfcfc4ef69952f1f6b28bfb84: Deleting consensus metadata
I20260620 08:13:15.034108 31435 catalog_manager.cc:4841] TS 880f394cfcfc4ef69952f1f6b28bfb84 (127.12.132.194:33241): tablet 44a7ea8fcf7b4842acdab4d81186921e (table test-table [id=bc3dcacdad1342419a49aeb2c472728f]) successfully deleted
W20260620 08:13:15.037030 31437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:36005 (error 108)
I20260620 08:13:15.037154 31938 tablet_replica.cc:330] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: stopping tablet replica
I20260620 08:13:15.037249 31938 raft_consensus.cc:2229] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:15.037357 31938 raft_consensus.cc:2258] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:15.037526 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:15.037381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:36005 (local address 127.0.0.1:48948)
0620 08:13:15.037456 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:15.037497 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:36005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260620 08:13:15.037631 31437 catalog_manager.cc:4584] TS fb1f4479fe92449f98a80208704ccb29 (127.12.132.193:36005): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44a7ea8fcf7b4842acdab4d81186921e: Network error: Client connection negotiation failed: client connection to 127.12.132.193:36005: connect: Connection refused (error 111)
I20260620 08:13:15.037935 31938 ts_tablet_manager.cc:1905] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:15.038427 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31643
I20260620 08:13:15.039868 31938 ts_tablet_manager.cc:1918] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:15.039938 31938 log.cc:1196] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/44a7ea8fcf7b4842acdab4d81186921e
I20260620 08:13:15.040172 31938 ts_tablet_manager.cc:1939] T 44a7ea8fcf7b4842acdab4d81186921e P f4657efa9edf4e46a045d0b6b9292544: Deleting consensus metadata
I20260620 08:13:15.040570 31435 catalog_manager.cc:4841] TS f4657efa9edf4e46a045d0b6b9292544 (127.12.132.195:42715): tablet 44a7ea8fcf7b4842acdab4d81186921e (table test-table [id=bc3dcacdad1342419a49aeb2c472728f]) successfully deleted
I20260620 08:13:15.043900 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31774
I20260620 08:13:15.049892 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31420
2026-06-20T08:13:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (692 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:15 dist-test-slave-z0sp krb5kdc[31944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:15 dist-test-slave-z0sp krb5kdc[31944](info): set up 2 sockets
Jun 20 08:13:15 dist-test-slave-z0sp krb5kdc[31944](info): commencing operation
krb5kdc: starting...
W20260620 08:13:16.096825 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:16.230698 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36069
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:16.312126 31957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:16.312338 31957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:16.312363 31957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:16.313586 31957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:16.313637 31957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:16.313651 31957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:16.313664 31957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:16.313676 31957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:16.315222 31957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:16.315470 31957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:16.315706 31957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:16.320783 31963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.320811 31962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:16.320803 31957 server_base.cc:1034] running on GCE node
W20260620 08:13:16.320761 31965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:16.321321 31957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:16.321650 31957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:16.322855 31957 hybrid_clock.cc:648] HybridClock initialized: now 1781943196322857 us; error 47 us; skew 500 ppm
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:16.326047 31957 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:16.326551 31957 webserver.cc:460] Webserver started at http://127.12.132.254:44245/ using document root <none> and password file <none>
I20260620 08:13:16.326778 31957 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:16.326851 31957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:16.327000 31957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:16.328101 31957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "270eea2634dc46faa426b932bd18b9ab"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "5fbe8177b5194e65befb6d9c89ca8227"
server_key_iv: "f6008e57ddfd2f3a62c21205761e6302"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.328459 31957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "270eea2634dc46faa426b932bd18b9ab"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "5fbe8177b5194e65befb6d9c89ca8227"
server_key_iv: "f6008e57ddfd2f3a62c21205761e6302"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.329835 31957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:16.330904 31972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.331171 31957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:16.331276 31957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "270eea2634dc46faa426b932bd18b9ab"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "5fbe8177b5194e65befb6d9c89ca8227"
server_key_iv: "f6008e57ddfd2f3a62c21205761e6302"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.331375 31957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:16.348376 31957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:16.350598 31957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:16.350781 31957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:16.354835 31957 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:36069
I20260620 08:13:16.354866 32024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:36069 every 8 connection(s)
I20260620 08:13:16.355188 31957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:16.355500 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 31957
I20260620 08:13:16.355639 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:16.355842 12819 external_mini_cluster.cc:1428] Setting key 7594ab5d9f33644f94d147b6a3e0a80d
I20260620 08:13:16.356163 32025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:16.359385 32025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: Bootstrap starting.
I20260620 08:13:16.360400 32025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:16.360759 32025 log.cc:824] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:16.361574 32025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: No bootstrap required, opened a new log
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:16.363919 32025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } }
I20260620 08:13:16.364095 32025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:16.364135 32025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270eea2634dc46faa426b932bd18b9ab, State: Initialized, Role: FOLLOWER
I20260620 08:13:16.364261 32025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } }
I20260620 08:13:16.364346 32025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:16.364368 32025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:16.364442 32025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:16.364985 32025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } }
I20260620 08:13:16.365103 32025 leader_election.cc:302] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 270eea2634dc46faa426b932bd18b9ab; no voters:
I20260620 08:13:16.365293 32025 leader_election.cc:288] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:16.365350 32030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:16.365510 32030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [term 1 LEADER]: Becoming Leader. State: Replica: 270eea2634dc46faa426b932bd18b9ab, State: Running, Role: LEADER
I20260620 08:13:16.365567 32025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:16.365670 32030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } }
I20260620 08:13:16.365978 32030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 270eea2634dc46faa426b932bd18b9ab. Latest consensus state: current_term: 1 leader_uuid: "270eea2634dc46faa426b932bd18b9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } } }
I20260620 08:13:16.366106 32030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:16.366103 32031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "270eea2634dc46faa426b932bd18b9ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270eea2634dc46faa426b932bd18b9ab" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36069 } } }
I20260620 08:13:16.366274 32031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:16.366329 32036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:16.366938 32036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:16.367415 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.356384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.12.132.254:36069)
0620 08:13:16.356586 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.356595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.356626 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:13:16.357289 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.357297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.357363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.357507 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:16.358083 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.359425 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.360194 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.360845 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.360965 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.363468 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:16.363473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:16.363473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:16.363487 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:16.365277 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.365708 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.365711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.365711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.365816 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.366048 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.366049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.366049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.366361 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:16.366527 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.367035 (+ 508us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.367254 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260620 08:13:16.368445 32036 catalog_manager.cc:1347] Generated new cluster ID: b49f16fbea0248a4bf8b30cb836bc45b
I20260620 08:13:16.368518 32036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:16.385166 32036 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:16.385766 32036 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:16.392207 32036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: Generated new TSK 0
I20260620 08:13:16.392549 32036 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.132.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
Entry for principal kudu/127.12.132.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab.
I20260620 08:13:16.428092 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:16.523377 32051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:16.523563 32051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:16.523586 32051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:16.524747 32051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:16.524787 32051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:16.524813 32051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:16.526207 32051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.193.keytab
--principal=kudu/127.12.132.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:16.526433 32051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:16.526669 32051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:16.531795 32057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.531766 32059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.531868 32056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:16.531932 32051 server_base.cc:1034] running on GCE node
I20260620 08:13:16.532208 32051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:16.532450 32051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:16.533629 32051 hybrid_clock.cc:648] HybridClock initialized: now 1781943196533613 us; error 34 us; skew 500 ppm
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:16.536346 32051 init.cc:377] Logged in from keytab as kudu/127.12.132.193@KRBTEST.COM (short username kudu)
I20260620 08:13:16.536803 32051 webserver.cc:460] Webserver started at http://127.12.132.193:33891/ using document root <none> and password file <none>
I20260620 08:13:16.537041 32051 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:16.537110 32051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:16.537257 32051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:16.538283 32051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "cb0b87213d1240d2ba55aae6c1593601"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "b97231256d2146ec13ae207123f13f60"
server_key_iv: "847eb8170d11b2e1901c282e5f49bad3"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.538651 32051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "cb0b87213d1240d2ba55aae6c1593601"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "b97231256d2146ec13ae207123f13f60"
server_key_iv: "847eb8170d11b2e1901c282e5f49bad3"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.539987 32051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:16.541066 32066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.541359 32051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:16.541476 32051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "cb0b87213d1240d2ba55aae6c1593601"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "b97231256d2146ec13ae207123f13f60"
server_key_iv: "847eb8170d11b2e1901c282e5f49bad3"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.541565 32051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:16.562117 32051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:16.564553 32051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:16.564734 32051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:16.565011 32051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:16.565320 32051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:16.565376 32051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.565418 32051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:16.565444 32051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.572631 32051 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:44531
I20260620 08:13:16.572674 32179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:44531 every 8 connection(s)
I20260620 08:13:16.572995 32051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:13:16.575100 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32051
I20260620 08:13:16.575242 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:16.575469 12819 external_mini_cluster.cc:1428] Setting key 93581b0f470b6cc639840a5b09db154a
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.193@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.132.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.194@KRBTEST.COM" created.
I20260620 08:13:16.587821 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.573875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:50961 (local address 127.12.132.254:36069)
0620 08:13:16.573972 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.573975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.574132 (+ 157us) server_negotiation.cc:407] Connection header received
0620 08:13:16.574661 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.574662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.574675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.574729 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:16.576003 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.576707 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.577693 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.577907 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.577984 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.580057 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:16.580058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:16.580059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:16.580066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:16.586310 (+ 6244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.586768 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.586769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.586769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.586832 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.587130 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.587131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.587131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.587274 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:16.587357 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.587640 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.587728 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 08:13:16.588013 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.573737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36069 (local address 127.12.132.193:50961)
0620 08:13:16.573993 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.574043 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.574555 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.574801 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.574804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.575618 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:16.575912 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.575917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.576811 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.576812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.577581 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.577583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.577991 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.577992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.578114 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.578631 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:16.578640 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:16.579958 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:16.586421 (+ 6463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.586430 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.586432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.586672 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.586943 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.586943 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.586944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.587052 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.587348 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:16.587363 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:16.587501 (+ 138us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.587608 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.587771 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
I20260620 08:13:16.588549 32180 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36069
I20260620 08:13:16.588670 32180 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:16.588871 32180 heartbeater.cc:511] Master 127.12.132.254:36069 requested a full tablet report, sending...
I20260620 08:13:16.589458 31989 ts_manager.cc:194] Registered new tserver with Master: cb0b87213d1240d2ba55aae6c1593601 (127.12.132.193:44531)
I20260620 08:13:16.590188 31989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.193@KRBTEST.COM'} at 127.12.132.193:50961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
Entry for principal kudu/127.12.132.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab.
I20260620 08:13:16.603403 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:16.683996 32185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:16.684190 32185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:16.684213 32185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:16.685393 32185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:16.685436 32185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:16.685461 32185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:16.686861 32185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.194.keytab
--principal=kudu/127.12.132.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:16.687072 32185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:16.687276 32185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:16.692147 32193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.692157 32190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.692157 32191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:16.692344 32185 server_base.cc:1034] running on GCE node
I20260620 08:13:16.692554 32185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:16.692796 32185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:16.693956 32185 hybrid_clock.cc:648] HybridClock initialized: now 1781943196693940 us; error 51 us; skew 500 ppm
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:16.696590 32185 init.cc:377] Logged in from keytab as kudu/127.12.132.194@KRBTEST.COM (short username kudu)
I20260620 08:13:16.697029 32185 webserver.cc:460] Webserver started at http://127.12.132.194:37145/ using document root <none> and password file <none>
I20260620 08:13:16.697252 32185 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:16.697324 32185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:16.697454 32185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:16.698455 32185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "1295c9598a06486c9b6f3b36adf2cc75"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "9ca17d83694df91d3f4cfd459200fe91"
server_key_iv: "30f9d495f0720047ccd8879affa272f9"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.698812 32185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "1295c9598a06486c9b6f3b36adf2cc75"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "9ca17d83694df91d3f4cfd459200fe91"
server_key_iv: "30f9d495f0720047ccd8879affa272f9"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.700102 32185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:16.700881 32200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.701046 32185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.701140 32185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "1295c9598a06486c9b6f3b36adf2cc75"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "9ca17d83694df91d3f4cfd459200fe91"
server_key_iv: "30f9d495f0720047ccd8879affa272f9"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.701220 32185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:16.731703 32185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:16.733978 32185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:16.734131 32185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:16.734380 32185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:16.734686 32185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:16.734740 32185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.734786 32185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:16.734807 32185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.740988 32185 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:34055
I20260620 08:13:16.741034 32313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:34055 every 8 connection(s)
I20260620 08:13:16.741334 32185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.194@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:16.749553 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32185
I20260620 08:13:16.749680 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:16.749914 12819 external_mini_cluster.cc:1428] Setting key b68b57a94367d3371566d76fb82ad4bb
I20260620 08:13:16.751099 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.742022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:36869 (local address 127.12.132.254:36069)
0620 08:13:16.742064 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.742066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.742263 (+ 197us) server_negotiation.cc:407] Connection header received
0620 08:13:16.742755 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.742756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.742767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.742831 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:16.743755 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.744435 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.745500 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.745700 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.745762 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.747925 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:16.747927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:16.747927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:16.747934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:16.749521 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.750076 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.750078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.750078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.750137 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.750410 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.750411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.750411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.750523 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:16.750607 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.750888 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.750970 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:16.751112 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.741984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36069 (local address 127.12.132.194:36869)
0620 08:13:16.742136 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.742175 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.742640 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.742903 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.742906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.743446 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:16.743631 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.743635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.744585 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.744586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.745402 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.745404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.745789 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.745790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.745894 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.746365 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:16.746373 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:16.747808 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:16.749664 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.749672 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.749674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.749992 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.750220 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.750221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.750221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.750341 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.750610 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:16.750616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:16.750747 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.750840 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.750957 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260620 08:13:16.752131 32314 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36069
I20260620 08:13:16.752259 32314 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:16.752440 32314 heartbeater.cc:511] Master 127.12.132.254:36069 requested a full tablet report, sending...
I20260620 08:13:16.752900 31989 ts_manager.cc:194] Registered new tserver with Master: 1295c9598a06486c9b6f3b36adf2cc75 (127.12.132.194:34055)
I20260620 08:13:16.753283 31989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.194@KRBTEST.COM'} at 127.12.132.194:36869
WARNING: no policy specified for kudu/127.12.132.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
Entry for principal kudu/127.12.132.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab.
I20260620 08:13:16.777359 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:16.859059 32319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:16.859282 32319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:16.859318 32319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:16.860414 32319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:16.860481 32319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:16.860550 32319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:16.861989 32319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.195.keytab
--principal=kudu/127.12.132.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:36069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:16.862254 32319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:16.862486 32319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:16.867884 32324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.867915 32325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:16.868302 32327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:16.868320 32319 server_base.cc:1034] running on GCE node
I20260620 08:13:16.868575 32319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:16.868805 32319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:16.870119 32319 hybrid_clock.cc:648] HybridClock initialized: now 1781943196870076 us; error 34 us; skew 500 ppm
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:16.873505 32319 init.cc:377] Logged in from keytab as kudu/127.12.132.195@KRBTEST.COM (short username kudu)
I20260620 08:13:16.874008 32319 webserver.cc:460] Webserver started at http://127.12.132.195:46347/ using document root <none> and password file <none>
I20260620 08:13:16.874235 32319 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:16.874302 32319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:16.874455 32319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:16.875487 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "337d513a801a4016a972e8d4468918e1"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "4537d442ffa94356f331cb47a5df9539"
server_key_iv: "861f6154a72f2545fab01d33eaf37f30"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.875859 32319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "337d513a801a4016a972e8d4468918e1"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "4537d442ffa94356f331cb47a5df9539"
server_key_iv: "861f6154a72f2545fab01d33eaf37f30"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.877869 32319 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 08:13:16.878793 32334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.878998 32319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:16.879105 32319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "337d513a801a4016a972e8d4468918e1"
format_stamp: "Formatted at 2026-06-20 08:13:16 on dist-test-slave-z0sp"
server_key: "4537d442ffa94356f331cb47a5df9539"
server_key_iv: "861f6154a72f2545fab01d33eaf37f30"
server_key_version: "encryptionkey@0"
I20260620 08:13:16.879194 32319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:16.899663 32319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:16.901974 32319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:16.902129 32319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:16.902361 32319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:16.902658 32319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:16.902712 32319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.902752 32319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:16.902778 32319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:16.909771 32319 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:36717
I20260620 08:13:16.909909 32447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:36717 every 8 connection(s)
I20260620 08:13:16.910156 32319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:13:16.913134 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32319
I20260620 08:13:16.913244 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:16.913403 12819 external_mini_cluster.cc:1428] Setting key 6f1dfe68d583697cd91be16d8ff5bf13
Jun 20 08:13:16 dist-test-slave-z0sp krb5kdc[31944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943196, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.195@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:16.920836 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.910952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:60949 (local address 127.12.132.254:36069)
0620 08:13:16.911034 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.911037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.911227 (+ 190us) server_negotiation.cc:407] Connection header received
0620 08:13:16.911793 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.911795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.911807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.911857 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:16.912807 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.913559 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.914706 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.914915 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.915001 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.917358 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:16.917359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:16.917360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:16.917367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:16.919001 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.919596 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.919597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.919598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.919670 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.919961 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.919962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.919963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.920190 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:16.920295 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.920625 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.920727 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:13:16.921066 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.910871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36069 (local address 127.12.132.195:60949)
0620 08:13:16.911078 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.911130 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.911648 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.911926 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.911930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.912486 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:16.912703 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.912708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.913701 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.913702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.914590 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.914594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.915014 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.915015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.915135 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.915587 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:16.915595 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:16.917227 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:16.919179 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.919195 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.919198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.919505 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.919763 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:16.919764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:16.919765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:16.919890 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:16.920296 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:16.920305 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:16.920448 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.920591 (+ 143us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.920802 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260620 08:13:16.921768 32448 heartbeater.cc:347] Connected to a master server at 127.12.132.254:36069
I20260620 08:13:16.921902 32448 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:16.922133 32448 heartbeater.cc:511] Master 127.12.132.254:36069 requested a full tablet report, sending...
I20260620 08:13:16.922514 31989 ts_manager.cc:194] Registered new tserver with Master: 337d513a801a4016a972e8d4468918e1 (127.12.132.195:36717)
I20260620 08:13:16.922904 31989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.132.195@KRBTEST.COM'} at 127.12.132.195:60949
I20260620 08:13:16.925316 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:16.933529 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.926658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.12.132.254:36069)
0620 08:13:16.926715 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.926717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.926862 (+ 145us) server_negotiation.cc:407] Connection header received
0620 08:13:16.926884 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.926885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.926894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.926954 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:16.927862 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.928339 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.929175 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.929358 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.929415 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.930603 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:16.930604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:16.930604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:16.930610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:16.932171 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.932618 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.932620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.932621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.932715 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:16.932990 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:16.932991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:16.932991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:16.933144 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:16.933229 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.933372 (+ 143us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.933435 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 08:13:16.935123 31989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:16.936709 31989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:16.943724 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:34055 (local address 127.0.0.1:57718)
0620 08:13:16.940153 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.940163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.940220 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.940405 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.940406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.940408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:16.940456 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.940457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941663 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.941664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.942577 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.942579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943466 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.943541 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.943546 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.943577 (+ 31us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.943622 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":74,"threads_started":1}
I20260620 08:13:16.943825 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:36717 (local address 127.0.0.1:55176)
0620 08:13:16.939591 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.939638 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.939707 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.939995 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.939998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.940000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:16.940066 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.940068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941407 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.941409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.942743 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.942745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943566 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.943650 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.943653 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.943697 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.943736 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":31,"thread_start_us":54,"threads_started":1}
I20260620 08:13:16.943836 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.12.132.195:36717)
0620 08:13:16.939755 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.939760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.939770 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:16.939806 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.939808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.939824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.939927 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:16.940143 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941283 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.942832 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943496 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943553 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.943603 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.943723 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.943765 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260620 08:13:16.944147 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.12.132.193:44531)
0620 08:13:16.940134 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.940138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.940358 (+ 220us) server_negotiation.cc:407] Connection header received
0620 08:13:16.940432 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.940435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.940451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.940586 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:16.940701 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941396 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.942854 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943319 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943410 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.943468 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.944015 (+ 547us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.944065 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260620 08:13:16.943714 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:44531 (local address 127.0.0.1:41784)
0620 08:13:16.940299 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:16.940306 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:16.940359 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:16.940586 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:16.940586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:16.940588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:16.940646 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.940647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941506 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.941507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.942771 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:16.942773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943410 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:16.943485 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.943498 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 08:13:16.943564 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:13:16.943614 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":29,"thread_start_us":54,"threads_started":1}
I20260620 08:13:16.944352 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:16.939754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57718 (local address 127.12.132.194:34055)
0620 08:13:16.939942 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:16.939946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:16.940215 (+ 269us) server_negotiation.cc:407] Connection header received
0620 08:13:16.940288 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:16.940291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:16.940306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:16.940547 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:16.940609 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.941563 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.942663 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:16.943366 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:16.943766 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:16.944031 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:16.944057 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 08:13:16.944099 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
I20260620 08:13:16.944561 32382 tablet_service.cc:1467] Processing CreateTablet for tablet 09f596c502a34261883300e8e537f923 (DEFAULT_TABLE table=test-table [id=9699fffb7ec44d8a9081c0d15de5286a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:16.944965 32382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f596c502a34261883300e8e537f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:16.944909 32114 tablet_service.cc:1467] Processing CreateTablet for tablet 09f596c502a34261883300e8e537f923 (DEFAULT_TABLE table=test-table [id=9699fffb7ec44d8a9081c0d15de5286a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:16.945183 32114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f596c502a34261883300e8e537f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:16.948179 32465 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Bootstrap starting.
I20260620 08:13:16.948132 32248 tablet_service.cc:1467] Processing CreateTablet for tablet 09f596c502a34261883300e8e537f923 (DEFAULT_TABLE table=test-table [id=9699fffb7ec44d8a9081c0d15de5286a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:16.948459 32248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f596c502a34261883300e8e537f923. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:16.948899 32465 tablet_bootstrap.cc:654] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:16.949326 32466 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Bootstrap starting.
I20260620 08:13:16.949429 32465 log.cc:824] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:16.950080 32466 tablet_bootstrap.cc:654] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:16.950289 32465 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: No bootstrap required, opened a new log
I20260620 08:13:16.950364 32465 ts_tablet_manager.cc:1397] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:16.950678 32466 log.cc:824] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:16.951516 32466 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: No bootstrap required, opened a new log
I20260620 08:13:16.951609 32466 ts_tablet_manager.cc:1397] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:16.952332 32465 raft_consensus.cc:348] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.952468 32465 raft_consensus.cc:374] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:16.952554 32465 raft_consensus.cc:729] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337d513a801a4016a972e8d4468918e1, State: Initialized, Role: FOLLOWER
I20260620 08:13:16.952648 32469 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Bootstrap starting.
I20260620 08:13:16.952695 32465 consensus_queue.cc:260] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.952991 32465 ts_tablet_manager.cc:1428] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 08:13:16.953117 32448 heartbeater.cc:503] Master 127.12.132.254:36069 was elected leader, sending a full tablet report...
I20260620 08:13:16.953228 32469 tablet_bootstrap.cc:654] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:16.953929 32469 log.cc:824] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:16.954605 32466 raft_consensus.cc:348] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.954829 32466 raft_consensus.cc:374] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:16.954871 32466 raft_consensus.cc:729] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0b87213d1240d2ba55aae6c1593601, State: Initialized, Role: FOLLOWER
I20260620 08:13:16.955030 32466 consensus_queue.cc:260] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.955240 32469 tablet_bootstrap.cc:492] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: No bootstrap required, opened a new log
I20260620 08:13:16.955376 32466 ts_tablet_manager.cc:1428] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 08:13:16.955418 32469 ts_tablet_manager.cc:1397] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:13:16.955533 32180 heartbeater.cc:503] Master 127.12.132.254:36069 was elected leader, sending a full tablet report...
I20260620 08:13:16.957268 32469 raft_consensus.cc:348] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.957422 32469 raft_consensus.cc:374] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:16.957444 32469 raft_consensus.cc:729] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1295c9598a06486c9b6f3b36adf2cc75, State: Initialized, Role: FOLLOWER
I20260620 08:13:16.957567 32469 consensus_queue.cc:260] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:16.957939 32469 ts_tablet_manager.cc:1428] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:16.957971 32314 heartbeater.cc:503] Master 127.12.132.254:36069 was elected leader, sending a full tablet report...
W20260620 08:13:16.991870 32315 tablet.cc:2369] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 08:13:17.073721 32181 tablet.cc:2369] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:17.134531 32470 raft_consensus.cc:482] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:17.134645 32470 raft_consensus.cc:504] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:17.135010 32470 leader_election.cc:288] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1295c9598a06486c9b6f3b36adf2cc75 (127.12.132.194:34055), cb0b87213d1240d2ba55aae6c1593601 (127.12.132.193:44531)
I20260620 08:13:17.138489 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.135289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:47845 (local address 127.12.132.194:34055)
0620 08:13:17.135338 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.135340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.135351 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:17.135412 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.135413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.135425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.135482 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.135725 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.136262 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.137685 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.138063 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.138128 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.138135 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.138367 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.138416 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:13:17.138535 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.135208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:34055 (local address 127.12.132.195:47845)
0620 08:13:17.135273 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.135287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.135350 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.135489 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.135490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.135492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.135597 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.135599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.136335 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.136336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.137328 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.137379 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.138181 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.138182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.138279 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.138283 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.138341 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.138411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":7}
I20260620 08:13:17.138573 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.135401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:44453 (local address 127.12.132.193:44531)
0620 08:13:17.135459 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.135461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.135554 (+ 93us) server_negotiation.cc:407] Connection header received
0620 08:13:17.135586 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.135587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.135596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.135662 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.135880 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.136367 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.137452 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.138176 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.138265 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.138278 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.138439 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.138505 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 08:13:17.138724 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.135328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:44531 (local address 127.12.132.195:44453)
0620 08:13:17.135477 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.135488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.135536 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.135718 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.135719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.135722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.135799 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.135801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.136446 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.136446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.137295 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.137297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.138272 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.138273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.138352 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.138357 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.138416 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.138451 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260620 08:13:17.138870 32268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f596c502a34261883300e8e537f923" candidate_uuid: "337d513a801a4016a972e8d4468918e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1295c9598a06486c9b6f3b36adf2cc75" is_pre_election: true
I20260620 08:13:17.139060 32268 raft_consensus.cc:2454] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337d513a801a4016a972e8d4468918e1 in term 0.
I20260620 08:13:17.139107 32134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f596c502a34261883300e8e537f923" candidate_uuid: "337d513a801a4016a972e8d4468918e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0b87213d1240d2ba55aae6c1593601" is_pre_election: true
I20260620 08:13:17.139251 32134 raft_consensus.cc:2454] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 337d513a801a4016a972e8d4468918e1 in term 0.
I20260620 08:13:17.139305 32336 leader_election.cc:302] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1295c9598a06486c9b6f3b36adf2cc75, 337d513a801a4016a972e8d4468918e1; no voters:
I20260620 08:13:17.139427 32470 raft_consensus.cc:2781] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:17.139505 32470 raft_consensus.cc:482] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:17.139539 32470 raft_consensus.cc:3037] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.140231 32470 raft_consensus.cc:504] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:17.140368 32470 leader_election.cc:288] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [CANDIDATE]: Term 1 election: Requested vote from peers 1295c9598a06486c9b6f3b36adf2cc75 (127.12.132.194:34055), cb0b87213d1240d2ba55aae6c1593601 (127.12.132.193:44531)
I20260620 08:13:17.140575 32268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f596c502a34261883300e8e537f923" candidate_uuid: "337d513a801a4016a972e8d4468918e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1295c9598a06486c9b6f3b36adf2cc75"
I20260620 08:13:17.140669 32268 raft_consensus.cc:3037] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.140662 32134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f596c502a34261883300e8e537f923" candidate_uuid: "337d513a801a4016a972e8d4468918e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb0b87213d1240d2ba55aae6c1593601"
I20260620 08:13:17.140767 32134 raft_consensus.cc:3037] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.141382 32268 raft_consensus.cc:2454] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337d513a801a4016a972e8d4468918e1 in term 1.
I20260620 08:13:17.141563 32134 raft_consensus.cc:2454] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 337d513a801a4016a972e8d4468918e1 in term 1.
I20260620 08:13:17.141577 32336 leader_election.cc:302] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1295c9598a06486c9b6f3b36adf2cc75, 337d513a801a4016a972e8d4468918e1; no voters:
I20260620 08:13:17.141669 32470 raft_consensus.cc:2781] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:17.141824 32470 raft_consensus.cc:686] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 1 LEADER]: Becoming Leader. State: Replica: 337d513a801a4016a972e8d4468918e1, State: Running, Role: LEADER
I20260620 08:13:17.141942 32470 consensus_queue.cc:237] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } }
I20260620 08:13:17.142760 31989 catalog_manager.cc:5495] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 337d513a801a4016a972e8d4468918e1 (127.12.132.195). New cstate: current_term: 1 leader_uuid: "337d513a801a4016a972e8d4468918e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337d513a801a4016a972e8d4468918e1" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 36717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 } health_report { overall_health: UNKNOWN } } }
W20260620 08:13:17.160686 32449 tablet.cc:2369] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:17.172307 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.169873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.12.132.195:36717)
0620 08:13:17.169919 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.169921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.169929 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:13:17.169966 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.169967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.169991 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.170067 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:17.170184 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.170647 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.171425 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.171710 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.171813 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.171977 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:17.172078 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:17.172173 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.172192 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.172219 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:13:17.174145 32268 raft_consensus.cc:1261] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 1 FOLLOWER]: Refusing update from remote peer 337d513a801a4016a972e8d4468918e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:17.174185 32134 raft_consensus.cc:1261] T 09f596c502a34261883300e8e537f923 P cb0b87213d1240d2ba55aae6c1593601 [term 1 FOLLOWER]: Refusing update from remote peer 337d513a801a4016a972e8d4468918e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:17.174535 32475 consensus_queue.cc:1035] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1295c9598a06486c9b6f3b36adf2cc75" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 34055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:17.174680 32470 consensus_queue.cc:1035] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb0b87213d1240d2ba55aae6c1593601" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 44531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:17.177014 32479 mvcc.cc:205] Tried to move back new op lower bound from 7298839335622496256 to 7298839335494066176. Current Snapshot: MvccSnapshot[applied={T|T < 7298839335622496256}]
I20260620 08:13:17.177852 32480 mvcc.cc:205] Tried to move back new op lower bound from 7298839335622496256 to 7298839335494066176. Current Snapshot: MvccSnapshot[applied={T|T < 7298839335622496256 or (T in {7298839335622496256})}]
I20260620 08:13:17.179600 31988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:17.179704 31988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:17.180483 31988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 270eea2634dc46faa426b932bd18b9ab: Sending DeleteTablet for 3 replicas of tablet 09f596c502a34261883300e8e537f923
I20260620 08:13:17.180858 32382 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f596c502a34261883300e8e537f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:17 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:55176
I20260620 08:13:17.180925 32114 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f596c502a34261883300e8e537f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:17 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:41784
I20260620 08:13:17.181072 32248 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f596c502a34261883300e8e537f923 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:17 UTC) from {username='kudu', principal='kudu/127.12.132.254@KRBTEST.COM'} at 127.0.0.1:57718
I20260620 08:13:17.181102 32487 tablet_replica.cc:330] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: stopping tablet replica
I20260620 08:13:17.181190 32487 raft_consensus.cc:2229] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:17.181306 32487 raft_consensus.cc:2258] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:17.181480 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32051
I20260620 08:13:17.182437 32487 ts_tablet_manager.cc:1905] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:17.183346 32489 tablet_replica.cc:330] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: stopping tablet replica
I20260620 08:13:17.183439 32489 raft_consensus.cc:2229] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:17.183532 32489 raft_consensus.cc:2258] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:17.184026 32489 ts_tablet_manager.cc:1905] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:17.184186 32487 ts_tablet_manager.cc:1918] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:17.184264 32487 log.cc:1196] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/09f596c502a34261883300e8e537f923
I20260620 08:13:17.184580 32487 ts_tablet_manager.cc:1939] T 09f596c502a34261883300e8e537f923 P 337d513a801a4016a972e8d4468918e1: Deleting consensus metadata
I20260620 08:13:17.184943 31974 catalog_manager.cc:4841] TS 337d513a801a4016a972e8d4468918e1 (127.12.132.195:36717): tablet 09f596c502a34261883300e8e537f923 (table test-table [id=9699fffb7ec44d8a9081c0d15de5286a]) successfully deleted
I20260620 08:13:17.185900 32489 ts_tablet_manager.cc:1918] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:17.186064 32489 log.cc:1196] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781942949664683-12819-0/minicluster-data/ts-1/wal/wals/09f596c502a34261883300e8e537f923
I20260620 08:13:17.186368 32489 ts_tablet_manager.cc:1939] T 09f596c502a34261883300e8e537f923 P 1295c9598a06486c9b6f3b36adf2cc75: Deleting consensus metadata
I20260620 08:13:17.186787 31974 catalog_manager.cc:4841] TS 1295c9598a06486c9b6f3b36adf2cc75 (127.12.132.194:34055): tablet 09f596c502a34261883300e8e537f923 (table test-table [id=9699fffb7ec44d8a9081c0d15de5286a]) successfully deleted
W20260620 08:13:17.188452 31974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:44531 (error 108)
I20260620 08:13:17.188791 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32185
I20260620 08:13:17.188997 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.188877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:44531 (local address 127.0.0.1:41794)
0620 08:13:17.188928 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.188961 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:44531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 08:13:17.189532 31974 catalog_manager.cc:4584] TS cb0b87213d1240d2ba55aae6c1593601 (127.12.132.193:44531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09f596c502a34261883300e8e537f923: Network error: Client connection negotiation failed: client connection to 127.12.132.193:44531: connect: Connection refused (error 111)
I20260620 08:13:17.194996 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32319
I20260620 08:13:17.199703 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 31957
2026-06-20T08:13:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2151 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T08:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:17Z Disabled control of system clock
I20260620 08:13:17.229048 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:34745
--webserver_interface=127.12.132.254
--webserver_port=0
--builtin_ntp_servers=127.12.132.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:34745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:17.309864 32494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:17.310074 32494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:17.310096 32494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:17.311156 32494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:17.311190 32494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:17.311206 32494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:17.311219 32494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:17.311230 32494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:17.312461 32494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:34745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:34745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:17.312657 32494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:17.312891 32494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:13:17.318078 32494 server_base.cc:1034] running on GCE node
W20260620 08:13:17.318043 32502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.318084 32499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.318043 32500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:17.318859 32494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:17.319152 32494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:17.320307 32494 hybrid_clock.cc:648] HybridClock initialized: now 1781943197320290 us; error 42 us; skew 500 ppm
I20260620 08:13:17.321691 32494 webserver.cc:460] Webserver started at http://127.12.132.254:34583/ using document root <none> and password file <none>
I20260620 08:13:17.321893 32494 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:17.321936 32494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:17.322021 32494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:17.322979 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "05e5d902faa84a0dab08324def23857b"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "a686c6aec5c23301d1d7cbb7ea8903ee"
server_key_iv: "f7d9d0802fc1166dd847dcc282c8d35c"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.323263 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "05e5d902faa84a0dab08324def23857b"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "a686c6aec5c23301d1d7cbb7ea8903ee"
server_key_iv: "f7d9d0802fc1166dd847dcc282c8d35c"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.324461 32494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:17.325189 32508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.325384 32494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 08:13:17.325470 32494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "05e5d902faa84a0dab08324def23857b"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "a686c6aec5c23301d1d7cbb7ea8903ee"
server_key_iv: "f7d9d0802fc1166dd847dcc282c8d35c"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.325524 32494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:17.339149 32494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:17.339367 32494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:17.339464 32494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:17.343204 32494 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:34745
I20260620 08:13:17.343321 32560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:34745 every 8 connection(s)
I20260620 08:13:17.343631 32494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:17.344851 32561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:17.347733 32561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: Bootstrap starting.
I20260620 08:13:17.348266 32561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:17.348608 32561 log.cc:824] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:17.349260 32561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: No bootstrap required, opened a new log
I20260620 08:13:17.350783 32561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } }
I20260620 08:13:17.350908 32561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:17.350930 32561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e5d902faa84a0dab08324def23857b, State: Initialized, Role: FOLLOWER
I20260620 08:13:17.351068 32561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } }
I20260620 08:13:17.351141 32561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:17.351162 32561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:17.351195 32561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.351642 32561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } }
I20260620 08:13:17.351722 32561 leader_election.cc:302] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05e5d902faa84a0dab08324def23857b; no voters:
I20260620 08:13:17.351874 32561 leader_election.cc:288] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:17.351913 32564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:17.352036 32564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [term 1 LEADER]: Becoming Leader. State: Replica: 05e5d902faa84a0dab08324def23857b, State: Running, Role: LEADER
I20260620 08:13:17.352123 32561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:17.352124 32564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } }
I20260620 08:13:17.352546 32565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05e5d902faa84a0dab08324def23857b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } } }
I20260620 08:13:17.352648 32565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:17.352628 32566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e5d902faa84a0dab08324def23857b. Latest consensus state: current_term: 1 leader_uuid: "05e5d902faa84a0dab08324def23857b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5d902faa84a0dab08324def23857b" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34745 } } }
I20260620 08:13:17.353001 32566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:17.353135 32573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:17.353638 32573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:17.353638 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32494
I20260620 08:13:17.353765 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:17.353951 12819 external_mini_cluster.cc:1428] Setting key 8cacec84efe8192bfbfde19dc0a329c4
I20260620 08:13:17.355180 32573 catalog_manager.cc:1347] Generated new cluster ID: afdfdaa9c9174372957fb3d8bc9f620c
I20260620 08:13:17.355233 32573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:17.358922 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.354391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.12.132.254:34745)
0620 08:13:17.354576 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.354584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.354613 (+ 29us) server_negotiation.cc:407] Connection header received
0620 08:13:17.354719 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.354725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.354799 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.354947 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:17.355188 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.356178 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.357046 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.357461 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.357593 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.358278 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:17.358282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:17.358282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:17.358453 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:17.358483 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:17.358493 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:17.358570 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.358642 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.358730 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260620 08:13:17.366732 32573 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:17.367306 32573 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:17.369613 32573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: Generated new TSK 0
I20260620 08:13:17.369771 32573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:17.377293 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.193:0
--local_ip_for_outbound_sockets=127.12.132.193
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--builtin_ntp_servers=127.12.132.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 08:13:17.465178 32585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:17.465371 32585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:17.465392 32585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:17.466485 32585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:17.466522 32585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:17.466547 32585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.193
I20260620 08:13:17.467844 32585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.132.193
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.193
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:17.468070 32585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:17.468369 32585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:17.474673 32591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:17.474704 32585 server_base.cc:1034] running on GCE node
W20260620 08:13:17.474794 32590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.474670 32593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:17.475263 32585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:17.475538 32585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:17.476688 32585 hybrid_clock.cc:648] HybridClock initialized: now 1781943197476676 us; error 29 us; skew 500 ppm
I20260620 08:13:17.478228 32585 webserver.cc:460] Webserver started at http://127.12.132.193:45835/ using document root <none> and password file <none>
I20260620 08:13:17.478487 32585 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:17.478551 32585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:17.478667 32585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:17.480088 32585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data/instance:
uuid: "c866f82fbacf4ff7b1c1dee055560c7d"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "8d10332ded23bdebbc50a21ca3f1a95b"
server_key_iv: "b597dd69883a235ef03c30b73b9288fb"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.480540 32585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance:
uuid: "c866f82fbacf4ff7b1c1dee055560c7d"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "8d10332ded23bdebbc50a21ca3f1a95b"
server_key_iv: "b597dd69883a235ef03c30b73b9288fb"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.482260 32585 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 08:13:17.483299 32599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.483615 32585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:17.483724 32585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal
uuid: "c866f82fbacf4ff7b1c1dee055560c7d"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "8d10332ded23bdebbc50a21ca3f1a95b"
server_key_iv: "b597dd69883a235ef03c30b73b9288fb"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.483816 32585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:17.504138 32585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:17.504443 32585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:17.504660 32585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:17.504878 32585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:17.505152 32585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:17.505182 32585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.505205 32585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:17.505218 32585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.512850 32585 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.193:41441
I20260620 08:13:17.512907 32712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.193:41441 every 8 connection(s)
I20260620 08:13:17.513226 32585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/data/info.pb
I20260620 08:13:17.513482 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32585
I20260620 08:13:17.513603 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-0/wal/instance
I20260620 08:13:17.513829 12819 external_mini_cluster.cc:1428] Setting key a73a1907c70997c1967a883689db8371
I20260620 08:13:17.514937 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.194:0
--local_ip_for_outbound_sockets=127.12.132.194
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--builtin_ntp_servers=127.12.132.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 08:13:17.521445 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.514655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.193:42311 (local address 127.12.132.254:34745)
0620 08:13:17.516249 (+ 1594us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.516251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.516259 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:13:17.516278 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.516279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.516296 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.516347 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:17.517406 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.518058 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.519751 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.520052 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.520124 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.521144 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:17.521145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:17.521146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:17.521150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:17.521171 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:17.521175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:17.521234 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.521334 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.521365 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1568}
I20260620 08:13:17.521660 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.514413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:34745 (local address 127.12.132.193:42311)
0620 08:13:17.514625 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.514697 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.515253 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.516474 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.516479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.516490 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:17.517230 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.517236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.518198 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.518199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.519634 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.519638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.520156 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.520157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.520331 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.520987 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:17.520987 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:17.521016 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:17.521038 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:17.521248 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:17.521253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:17.521256 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.521322 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.521422 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260620 08:13:17.522279 32713 heartbeater.cc:347] Connected to a master server at 127.12.132.254:34745
I20260620 08:13:17.522372 32713 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:17.522557 32713 heartbeater.cc:511] Master 127.12.132.254:34745 requested a full tablet report, sending...
I20260620 08:13:17.523051 32525 ts_manager.cc:194] Registered new tserver with Master: c866f82fbacf4ff7b1c1dee055560c7d (127.12.132.193:41441)
I20260620 08:13:17.523676 32525 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.193:42311
W20260620 08:13:17.601135 32716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:17.601356 32716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:17.601389 32716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:17.602488 32716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:17.602551 32716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:17.602594 32716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.194
I20260620 08:13:17.603943 32716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.132.194
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.194
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:17.604205 32716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:17.604537 32716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:17.609678 32722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.609676 32724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.609673 32721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:17.609916 32716 server_base.cc:1034] running on GCE node
I20260620 08:13:17.610139 32716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:17.610375 32716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:17.611541 32716 hybrid_clock.cc:648] HybridClock initialized: now 1781943197611517 us; error 40 us; skew 500 ppm
I20260620 08:13:17.612914 32716 webserver.cc:460] Webserver started at http://127.12.132.194:46207/ using document root <none> and password file <none>
I20260620 08:13:17.613148 32716 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:17.613222 32716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:17.613376 32716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:17.614393 32716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data/instance:
uuid: "700f509c77fd46a1968fa3f6e1d45757"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "ec03a0112128982a0cba5ce7843b2bcd"
server_key_iv: "bc9698cfdacd99b9be01f37a871c7fda"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.614743 32716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance:
uuid: "700f509c77fd46a1968fa3f6e1d45757"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "ec03a0112128982a0cba5ce7843b2bcd"
server_key_iv: "bc9698cfdacd99b9be01f37a871c7fda"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.616144 32716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:17.616930 32730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.617096 32716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.617192 32716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal
uuid: "700f509c77fd46a1968fa3f6e1d45757"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "ec03a0112128982a0cba5ce7843b2bcd"
server_key_iv: "bc9698cfdacd99b9be01f37a871c7fda"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.617273 32716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:17.648509 32716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:17.648844 32716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:17.648999 32716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:17.649246 32716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:17.649577 32716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:17.649631 32716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.649674 32716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:17.649713 32716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.656020 32716 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.194:40241
I20260620 08:13:17.656069 375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.194:40241 every 8 connection(s)
I20260620 08:13:17.656384 32716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/data/info.pb
I20260620 08:13:17.660856 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 32716
I20260620 08:13:17.661082 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-1/wal/instance
I20260620 08:13:17.661406 12819 external_mini_cluster.cc:1428] Setting key c6298a3b0b02b200269076cdae1101e7
I20260620 08:13:17.662186 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.657129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:33737 (local address 127.12.132.254:34745)
0620 08:13:17.657178 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.657180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.657393 (+ 213us) server_negotiation.cc:407] Connection header received
0620 08:13:17.657951 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.657952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.657964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.658016 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:17.658700 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.659210 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.660190 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.660389 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.660477 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.661758 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:17.661761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:17.661761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:17.661770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:17.661797 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:17.661804 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:17.661881 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.662061 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.662103 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:17.662473 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.657092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:34745 (local address 127.12.132.194:33737)
0620 08:13:17.657253 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.657293 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.657838 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.658092 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.658096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.658105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:17.658606 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.658609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.659307 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.659308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.660096 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.660098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.660509 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.660511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.660654 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.661442 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:17.661443 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:17.661510 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:17.661556 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:17.661899 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:17.661907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:17.661911 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.662007 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.662137 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260620 08:13:17.662976 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.132.195:0
--local_ip_for_outbound_sockets=127.12.132.195
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--builtin_ntp_servers=127.12.132.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 08:13:17.662983 376 heartbeater.cc:347] Connected to a master server at 127.12.132.254:34745
I20260620 08:13:17.663094 376 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:17.663291 376 heartbeater.cc:511] Master 127.12.132.254:34745 requested a full tablet report, sending...
I20260620 08:13:17.663869 32525 ts_manager.cc:194] Registered new tserver with Master: 700f509c77fd46a1968fa3f6e1d45757 (127.12.132.194:40241)
I20260620 08:13:17.664324 32525 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.194:33737
W20260620 08:13:17.747347 379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:17.747538 379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:17.747560 379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:17.748673 379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:17.748716 379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:17.748741 379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.132.195
I20260620 08:13:17.750041 379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.132.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.132.195
--webserver_port=0
--tserver_master_addrs=127.12.132.254:34745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.132.195
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:17.750299 379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:17.750587 379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:17.755859 387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.755985 385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:17.755862 384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:17.756436 379 server_base.cc:1034] running on GCE node
I20260620 08:13:17.756711 379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:17.756963 379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:17.758128 379 hybrid_clock.cc:648] HybridClock initialized: now 1781943197758109 us; error 39 us; skew 500 ppm
I20260620 08:13:17.759729 379 webserver.cc:460] Webserver started at http://127.12.132.195:36297/ using document root <none> and password file <none>
I20260620 08:13:17.760102 379 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:17.760164 379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:17.760330 379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:17.761704 379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data/instance:
uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "29f98897ebec531b4d0f6a0f814e5ecf"
server_key_iv: "18390a25a9a119a9ba0680409625a0b3"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.762125 379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance:
uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "29f98897ebec531b4d0f6a0f814e5ecf"
server_key_iv: "18390a25a9a119a9ba0680409625a0b3"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.763854 379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:17.765174 393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.765492 379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:17.765617 379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal
uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6"
format_stamp: "Formatted at 2026-06-20 08:13:17 on dist-test-slave-z0sp"
server_key: "29f98897ebec531b4d0f6a0f814e5ecf"
server_key_iv: "18390a25a9a119a9ba0680409625a0b3"
server_key_version: "encryptionkey@0"
I20260620 08:13:17.765717 379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:17.793977 379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:17.794451 379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:17.794632 379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:17.794924 379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 08:13:17.795344 379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 08:13:17.795405 379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.795445 379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 08:13:17.795475 379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:17.803431 379 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.195:42047
I20260620 08:13:17.803467 506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.195:42047 every 8 connection(s)
I20260620 08:13:17.803862 379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/data/info.pb
I20260620 08:13:17.809680 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 379
I20260620 08:13:17.809815 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal/instance
I20260620 08:13:17.809944 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.804714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.195:47131 (local address 127.12.132.254:34745)
0620 08:13:17.804757 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.804759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.805009 (+ 250us) server_negotiation.cc:407] Connection header received
0620 08:13:17.805906 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.805908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.805918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.806041 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:17.806932 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.807438 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.808434 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.808665 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.808728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.809568 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:17.809570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:17.809570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:17.809574 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:17.809588 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:17.809592 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:17.809639 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.809795 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.809825 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:13:17.810045 12819 external_mini_cluster.cc:1428] Setting key 03d3a2bdc1c6793167254025ab6474e5
I20260620 08:13:17.810087 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.804695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:34745 (local address 127.12.132.195:47131)
0620 08:13:17.804884 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.804928 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.805721 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.806250 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.806256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.806267 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:17.806809 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.806815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.807559 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.807560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.808348 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.808350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.808747 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.808748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.808860 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.809393 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 08:13:17.809394 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:17.809421 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 08:13:17.809471 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:17.809656 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:17.809662 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:17.809665 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.809739 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.809861 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260620 08:13:17.810755 507 heartbeater.cc:347] Connected to a master server at 127.12.132.254:34745
I20260620 08:13:17.810860 507 heartbeater.cc:464] Registering TS with master...
I20260620 08:13:17.811050 507 heartbeater.cc:511] Master 127.12.132.254:34745 requested a full tablet report, sending...
I20260620 08:13:17.811452 32525 ts_manager.cc:194] Registered new tserver with Master: cf7dae3f4ed04f5abe1d50e5d17170f6 (127.12.132.195:42047)
I20260620 08:13:17.811800 32525 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.132.195:47131
I20260620 08:13:17.821903 12819 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 08:13:17.826522 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.823412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.12.132.254:34745)
0620 08:13:17.823462 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.823463 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.823528 (+ 65us) server_negotiation.cc:407] Connection header received
0620 08:13:17.823581 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.823581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.823590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.823659 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:17.824008 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.824478 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.825269 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.825445 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.825506 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.826316 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:17.826318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:17.826318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:17.826322 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:17.826333 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:17.826337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:17.826389 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.826452 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.826471 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:13:17.828461 32525 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 08:13:17.830621 32525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 08:13:17.837952 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:40241 (local address 127.0.0.1:41390)
0620 08:13:17.833450 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.833489 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.833556 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.834175 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.834178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.834181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.834275 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.834277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835730 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.835731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.836945 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.836948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837647 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.837726 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.837738 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.837823 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.837878 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260620 08:13:17.837962 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.12.132.194:40241)
0620 08:13:17.833925 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.833929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.833940 (+ 11us) server_negotiation.cc:407] Connection header received
0620 08:13:17.833983 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.833985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.834001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.834424 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.834489 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835625 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837061 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837551 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837650 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.837743 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.837837 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.837875 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":135,"threads_started":1}
I20260620 08:13:17.838174 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42047 (local address 127.0.0.1:45844)
0620 08:13:17.833684 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.833697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.833772 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.834292 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.834293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.834303 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.834337 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.834338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835446 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.835447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.836951 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.836953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837952 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.838014 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.838017 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.838061 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.838100 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":23,"spinlock_wait_cycles":768,"thread_start_us":132,"threads_started":1}
I20260620 08:13:17.838196 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.12.132.193:41441)
0620 08:13:17.833783 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.833788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.834017 (+ 229us) server_negotiation.cc:407] Connection header received
0620 08:13:17.834181 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.834184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.834206 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.834268 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.834577 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835714 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.836962 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837677 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837768 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.837854 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.838057 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.838098 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":143,"threads_started":1}
I20260620 08:13:17.838478 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.12.132.195:42047)
0620 08:13:17.834074 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.834081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.834090 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:17.834145 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.834147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.834166 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.834272 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.834493 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835313 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837051 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837775 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837850 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.837923 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.838358 (+ 435us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.838402 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260620 08:13:17.838413 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.833474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:41441 (local address 127.0.0.1:34476)
0620 08:13:17.833865 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.833875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.834060 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.834384 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.834385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.834389 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.834461 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.834464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.835788 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.835790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.836872 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.836875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.837891 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.837892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.837959 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.837963 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.838307 (+ 344us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.838355 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":112,"threads_started":1}
I20260620 08:13:17.838843 309 tablet_service.cc:1467] Processing CreateTablet for tablet 923af59ced0048658815dec920c3d6c2 (DEFAULT_TABLE table=test-table [id=b7222cbc1b6b482a9804c13d188b4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:17.839005 441 tablet_service.cc:1467] Processing CreateTablet for tablet 923af59ced0048658815dec920c3d6c2 (DEFAULT_TABLE table=test-table [id=b7222cbc1b6b482a9804c13d188b4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:17.839262 309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923af59ced0048658815dec920c3d6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:17.839367 441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923af59ced0048658815dec920c3d6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:17.841958 521 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Bootstrap starting.
I20260620 08:13:17.842361 522 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Bootstrap starting.
I20260620 08:13:17.842593 32647 tablet_service.cc:1467] Processing CreateTablet for tablet 923af59ced0048658815dec920c3d6c2 (DEFAULT_TABLE table=test-table [id=b7222cbc1b6b482a9804c13d188b4b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 08:13:17.842828 521 tablet_bootstrap.cc:654] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:17.842917 32647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923af59ced0048658815dec920c3d6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:17.842964 522 tablet_bootstrap.cc:654] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:17.843165 521 log.cc:824] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:17.843264 522 log.cc:824] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:17.844436 521 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: No bootstrap required, opened a new log
I20260620 08:13:17.844555 521 ts_tablet_manager.cc:1397] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 08:13:17.844864 522 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: No bootstrap required, opened a new log
I20260620 08:13:17.844945 522 ts_tablet_manager.cc:1397] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 08:13:17.846758 522 raft_consensus.cc:348] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.846922 522 raft_consensus.cc:374] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:17.846966 522 raft_consensus.cc:729] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7dae3f4ed04f5abe1d50e5d17170f6, State: Initialized, Role: FOLLOWER
I20260620 08:13:17.847218 525 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Bootstrap starting.
I20260620 08:13:17.847198 522 consensus_queue.cc:260] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.847461 522 ts_tablet_manager.cc:1428] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 08:13:17.847599 507 heartbeater.cc:503] Master 127.12.132.254:34745 was elected leader, sending a full tablet report...
I20260620 08:13:17.847486 521 raft_consensus.cc:348] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.847687 521 raft_consensus.cc:374] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:17.847739 521 raft_consensus.cc:729] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 700f509c77fd46a1968fa3f6e1d45757, State: Initialized, Role: FOLLOWER
I20260620 08:13:17.847851 521 consensus_queue.cc:260] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.847963 525 tablet_bootstrap.cc:654] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:17.848107 521 ts_tablet_manager.cc:1428] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 08:13:17.848181 376 heartbeater.cc:503] Master 127.12.132.254:34745 was elected leader, sending a full tablet report...
I20260620 08:13:17.848301 525 log.cc:824] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:17.849067 525 tablet_bootstrap.cc:492] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: No bootstrap required, opened a new log
I20260620 08:13:17.849117 525 ts_tablet_manager.cc:1397] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 08:13:17.850878 525 raft_consensus.cc:348] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.851032 525 raft_consensus.cc:374] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:17.851078 525 raft_consensus.cc:729] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c866f82fbacf4ff7b1c1dee055560c7d, State: Initialized, Role: FOLLOWER
I20260620 08:13:17.851178 525 consensus_queue.cc:260] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.851430 525 ts_tablet_manager.cc:1428] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 08:13:17.851516 32713 heartbeater.cc:503] Master 127.12.132.254:34745 was elected leader, sending a full tablet report...
W20260620 08:13:17.906986 377 tablet.cc:2369] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:17.986650 527 raft_consensus.cc:482] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 08:13:17.986830 527 raft_consensus.cc:504] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.987360 527 leader_election.cc:288] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c866f82fbacf4ff7b1c1dee055560c7d (127.12.132.193:41441), cf7dae3f4ed04f5abe1d50e5d17170f6 (127.12.132.195:42047)
I20260620 08:13:17.990492 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.987626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:41441 (local address 127.12.132.194:44769)
0620 08:13:17.987695 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.987722 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.987775 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.988038 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.988039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.988041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.988122 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.988124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.988899 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.988900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.989674 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.989675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.990228 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.990229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.990301 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.990304 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.990372 (+ 68us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.990416 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 08:13:17.990492 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.987808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:44769 (local address 127.12.132.193:41441)
0620 08:13:17.987896 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.987899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.987911 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:17.987945 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.987945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.987958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.988033 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.988205 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.988812 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.989779 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.990127 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.990227 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.990236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.990378 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.990415 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 08:13:17.990882 32667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923af59ced0048658815dec920c3d6c2" candidate_uuid: "700f509c77fd46a1968fa3f6e1d45757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" is_pre_election: true
I20260620 08:13:17.991035 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.987626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.195:42047 (local address 127.12.132.194:41149)
0620 08:13:17.987932 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:17.987945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:17.988078 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:17.988255 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:17.988256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:17.988258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 08:13:17.988361 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.988363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.989315 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.989317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.990083 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:17.990085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.990779 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.990780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:17.990844 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.990847 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 08:13:17.990914 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 08:13:17.990955 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":60,"thread_start_us":100,"threads_started":1}
I20260620 08:13:17.991050 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:17.987835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.132.194:41149 (local address 127.12.132.195:42047)
0620 08:13:17.987922 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:17.987925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:17.988061 (+ 136us) server_negotiation.cc:407] Connection header received
0620 08:13:17.988165 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:17.988166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:17.988183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:17.988250 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 08:13:17.988434 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.989178 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.990197 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:17.990632 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:17.990726 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:17.990734 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:17.990940 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 08:13:17.990986 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 08:13:17.991070 32667 raft_consensus.cc:2454] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 700f509c77fd46a1968fa3f6e1d45757 in term 0.
I20260620 08:13:17.991331 32733 leader_election.cc:302] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 700f509c77fd46a1968fa3f6e1d45757, c866f82fbacf4ff7b1c1dee055560c7d; no voters:
I20260620 08:13:17.991469 527 raft_consensus.cc:2781] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 08:13:17.991526 527 raft_consensus.cc:482] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 08:13:17.991544 527 raft_consensus.cc:3037] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.991473 461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923af59ced0048658815dec920c3d6c2" candidate_uuid: "700f509c77fd46a1968fa3f6e1d45757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" is_pre_election: true
I20260620 08:13:17.991623 461 raft_consensus.cc:2454] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 700f509c77fd46a1968fa3f6e1d45757 in term 0.
I20260620 08:13:17.992350 527 raft_consensus.cc:504] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.992501 527 leader_election.cc:288] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [CANDIDATE]: Term 1 election: Requested vote from peers c866f82fbacf4ff7b1c1dee055560c7d (127.12.132.193:41441), cf7dae3f4ed04f5abe1d50e5d17170f6 (127.12.132.195:42047)
I20260620 08:13:17.992645 461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923af59ced0048658815dec920c3d6c2" candidate_uuid: "700f509c77fd46a1968fa3f6e1d45757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6"
I20260620 08:13:17.992730 461 raft_consensus.cc:3037] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.992786 32667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923af59ced0048658815dec920c3d6c2" candidate_uuid: "700f509c77fd46a1968fa3f6e1d45757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c866f82fbacf4ff7b1c1dee055560c7d"
I20260620 08:13:17.992889 32667 raft_consensus.cc:3037] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:17.993408 461 raft_consensus.cc:2454] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 700f509c77fd46a1968fa3f6e1d45757 in term 1.
I20260620 08:13:17.993608 32734 leader_election.cc:302] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 700f509c77fd46a1968fa3f6e1d45757, cf7dae3f4ed04f5abe1d50e5d17170f6; no voters:
I20260620 08:13:17.993706 527 raft_consensus.cc:2781] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:17.993847 527 raft_consensus.cc:686] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 1 LEADER]: Becoming Leader. State: Replica: 700f509c77fd46a1968fa3f6e1d45757, State: Running, Role: LEADER
I20260620 08:13:17.993888 32667 raft_consensus.cc:2454] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 700f509c77fd46a1968fa3f6e1d45757 in term 1.
I20260620 08:13:17.993974 527 consensus_queue.cc:237] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } }
I20260620 08:13:17.994817 32524 catalog_manager.cc:5495] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 reported cstate change: term changed from 0 to 1, leader changed from <none> to 700f509c77fd46a1968fa3f6e1d45757 (127.12.132.194). New cstate: current_term: 1 leader_uuid: "700f509c77fd46a1968fa3f6e1d45757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "700f509c77fd46a1968fa3f6e1d45757" member_type: VOTER last_known_addr { host: "127.12.132.194" port: 40241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 } health_report { overall_health: UNKNOWN } } }
W20260620 08:13:18.013968 32714 tablet.cc:2369] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 08:13:18.022645 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:18.020138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.12.132.194:40241)
0620 08:13:18.020203 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:18.020205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:18.020213 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:13:18.020255 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:18.020256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:18.020276 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:18.020380 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:18.020530 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:18.021014 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:18.021764 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:18.022062 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:18.022158 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:18.022303 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:18.022412 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:18.022507 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:18.022526 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 08:13:18.022544 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:13:18.024786 32667 raft_consensus.cc:1261] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d [term 1 FOLLOWER]: Refusing update from remote peer 700f509c77fd46a1968fa3f6e1d45757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:18.024871 461 raft_consensus.cc:1261] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 1 FOLLOWER]: Refusing update from remote peer 700f509c77fd46a1968fa3f6e1d45757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:18.025174 527 consensus_queue.cc:1035] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c866f82fbacf4ff7b1c1dee055560c7d" member_type: VOTER last_known_addr { host: "127.12.132.193" port: 41441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:18.025370 531 consensus_queue.cc:1035] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7dae3f4ed04f5abe1d50e5d17170f6" member_type: VOTER last_known_addr { host: "127.12.132.195" port: 42047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:18.027501 534 mvcc.cc:205] Tried to move back new op lower bound from 7298839339106111488 to 7298839338983882752. Current Snapshot: MvccSnapshot[applied={T|T < 7298839339106111488}]
I20260620 08:13:18.027853 536 mvcc.cc:205] Tried to move back new op lower bound from 7298839339106111488 to 7298839338983882752. Current Snapshot: MvccSnapshot[applied={T|T < 7298839339106111488}]
I20260620 08:13:18.028228 535 mvcc.cc:205] Tried to move back new op lower bound from 7298839339106111488 to 7298839338983882752. Current Snapshot: MvccSnapshot[applied={T|T < 7298839339106111488 or (T in {7298839339106111488})}]
I20260620 08:13:18.030018 32524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:18.030098 32524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 08:13:18.030876 32524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05e5d902faa84a0dab08324def23857b: Sending DeleteTablet for 3 replicas of tablet 923af59ced0048658815dec920c3d6c2
I20260620 08:13:18.031208 441 tablet_service.cc:1514] Processing DeleteTablet for tablet 923af59ced0048658815dec920c3d6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:18 UTC) from {username='slave'} at 127.0.0.1:45844
I20260620 08:13:18.031208 309 tablet_service.cc:1514] Processing DeleteTablet for tablet 923af59ced0048658815dec920c3d6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:18 UTC) from {username='slave'} at 127.0.0.1:41390
I20260620 08:13:18.031299 32647 tablet_service.cc:1514] Processing DeleteTablet for tablet 923af59ced0048658815dec920c3d6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 08:13:18 UTC) from {username='slave'} at 127.0.0.1:34476
I20260620 08:13:18.031450 544 tablet_replica.cc:330] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: stopping tablet replica
I20260620 08:13:18.031471 546 tablet_replica.cc:330] T 923af59ced0048658815dec920c3d6c2 P c866f82fbacf4ff7b1c1dee055560c7d: stopping tablet replica
I20260620 08:13:18.031546 544 raft_consensus.cc:2229] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 08:13:18.031652 544 raft_consensus.cc:2258] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:18.031781 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32585
I20260620 08:13:18.032055 544 ts_tablet_manager.cc:1905] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 08:13:18.034133 544 ts_tablet_manager.cc:1918] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 08:13:18.034216 544 log.cc:1196] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Deleting WAL directory at /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781942949664683-12819-0/minicluster-data/ts-2/wal/wals/923af59ced0048658815dec920c3d6c2
I20260620 08:13:18.034456 544 ts_tablet_manager.cc:1939] T 923af59ced0048658815dec920c3d6c2 P cf7dae3f4ed04f5abe1d50e5d17170f6: Deleting consensus metadata
I20260620 08:13:18.034826 32512 catalog_manager.cc:4841] TS cf7dae3f4ed04f5abe1d50e5d17170f6 (127.12.132.195:42047): tablet 923af59ced0048658815dec920c3d6c2 (table test-table [id=b7222cbc1b6b482a9804c13d188b4b49]) successfully deleted
W20260620 08:13:18.038676 32511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.132.193:41441 (error 108)
I20260620 08:13:18.038712 545 tablet_replica.cc:330] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757: stopping tablet replica
I20260620 08:13:18.038893 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32716
I20260620 08:13:18.038918 545 raft_consensus.cc:2229] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 1 LEADER]: Raft consensus shutting down.
I20260620 08:13:18.039119 545 raft_consensus.cc:2258] T 923af59ced0048658815dec920c3d6c2 P 700f509c77fd46a1968fa3f6e1d45757 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 08:13:18.039566 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:18.039020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.193:41441 (local address 127.0.0.1:34482)
0620 08:13:18.039488 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:18.039529 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.193:41441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447}
W20260620 08:13:18.039861 32511 catalog_manager.cc:4584] TS c866f82fbacf4ff7b1c1dee055560c7d (127.12.132.193:41441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 923af59ced0048658815dec920c3d6c2: Network error: Client connection negotiation failed: client connection to 127.12.132.193:41441: connect: Connection refused (error 111)
I20260620 08:13:18.043390 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:18.043266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.194:40241 (local address 127.0.0.1:41404)
0620 08:13:18.043339 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:18.043369 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.194:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 08:13:18.043399 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 379
I20260620 08:13:18.048363 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 32494
2026-06-20T08:13:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (847 ms)
[----------] 6 tests from EncryptionPolicyTest (8954 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:18 dist-test-slave-z0sp krb5kdc[550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:18 dist-test-slave-z0sp krb5kdc[550](info): set up 2 sockets
Jun 20 08:13:18 dist-test-slave-z0sp krb5kdc[550](info): commencing operation
krb5kdc: starting...
W20260620 08:13:19.100936 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:19.227633 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:19.310539 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:19.310786 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:19.310812 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:19.312088 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:19.312146 563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:19.312161 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:19.312173 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:19.312186 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:19.313872 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:19.314136 563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:19.314400 563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:19.319625 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:19.319636 569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:19.319859 568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:19.319990 563 server_base.cc:1034] running on GCE node
I20260620 08:13:19.320199 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:19.320485 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:19.321719 563 hybrid_clock.cc:648] HybridClock initialized: now 1781943199321698 us; error 48 us; skew 500 ppm
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:19.324435 563 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:19.325018 563 webserver.cc:460] Webserver started at http://127.0.0.1:42413/ using document root <none> and password file <none>
I20260620 08:13:19.325317 563 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:19.325369 563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:19.325500 563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:19.326555 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ddda055968484049936d99078673ecf1"
format_stamp: "Formatted at 2026-06-20 08:13:19 on dist-test-slave-z0sp"
server_key: "b5417f57b6a448cec213bbbea3ad977d"
server_key_iv: "b10554d3034b0a5a7cd3dd806f1de3e1"
server_key_version: "encryptionkey@0"
I20260620 08:13:19.326905 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ddda055968484049936d99078673ecf1"
format_stamp: "Formatted at 2026-06-20 08:13:19 on dist-test-slave-z0sp"
server_key: "b5417f57b6a448cec213bbbea3ad977d"
server_key_iv: "b10554d3034b0a5a7cd3dd806f1de3e1"
server_key_version: "encryptionkey@0"
I20260620 08:13:19.328253 563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:19.329073 578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:19.329285 563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:19.329381 563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ddda055968484049936d99078673ecf1"
format_stamp: "Formatted at 2026-06-20 08:13:19 on dist-test-slave-z0sp"
server_key: "b5417f57b6a448cec213bbbea3ad977d"
server_key_iv: "b10554d3034b0a5a7cd3dd806f1de3e1"
server_key_version: "encryptionkey@0"
I20260620 08:13:19.329439 563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:19.342809 563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:19.344916 563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:19.345026 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:19.348765 563 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45049
I20260620 08:13:19.348899 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45049 every 8 connection(s)
I20260620 08:13:19.349201 563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:19.350064 631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:19.352523 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 563
I20260620 08:13:19.352654 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:19.352711 631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1: Bootstrap starting.
I20260620 08:13:19.352876 12819 external_mini_cluster.cc:1428] Setting key 9f6b557d9c8e62e4e83991948987bd57
I20260620 08:13:19.353410 631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:19.353775 631 log.cc:824] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:19.354501 631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1: No bootstrap required, opened a new log
I20260620 08:13:19.356796 631 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } }
I20260620 08:13:19.356966 631 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:19.357014 631 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddda055968484049936d99078673ecf1, State: Initialized, Role: FOLLOWER
I20260620 08:13:19.357158 631 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } }
I20260620 08:13:19.357247 631 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:19.357291 631 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:19.357350 631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:19.358016 631 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } }
I20260620 08:13:19.358117 631 leader_election.cc:302] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddda055968484049936d99078673ecf1; no voters:
I20260620 08:13:19.358254 631 leader_election.cc:288] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:19.358275 636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:19.358404 636 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [term 1 LEADER]: Becoming Leader. State: Replica: ddda055968484049936d99078673ecf1, State: Running, Role: LEADER
I20260620 08:13:19.358496 636 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } }
I20260620 08:13:19.358531 631 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:19.358971 636 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddda055968484049936d99078673ecf1. Latest consensus state: current_term: 1 leader_uuid: "ddda055968484049936d99078673ecf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } } }
I20260620 08:13:19.359069 636 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:19.359027 637 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddda055968484049936d99078673ecf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddda055968484049936d99078673ecf1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45049 } } }
I20260620 08:13:19.359416 637 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:19.360164 645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 08:13:19.361037 645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:19.363041 645 catalog_manager.cc:1347] Generated new cluster ID: 57c9dccbda1d42489deb515ec48f6306
I20260620 08:13:19.363117 645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:19.366055 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:19.353388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47217 (local address 127.0.0.1:45049)
0620 08:13:19.353642 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:19.353651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:19.353682 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:13:19.354235 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:19.354243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:19.354302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:19.354439 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:19.355069 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:19.356533 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:19.357344 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:19.357958 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:19.358074 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:19.361653 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:19.361657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:19.361658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:19.361671 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:19.363917 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:19.364394 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:19.364397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:19.364397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:19.364533 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:19.364788 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:19.364789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:19.364790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:19.365100 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:19.365268 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:19.365723 (+ 455us) server_negotiation.cc:299] Negotiation successful
0620 08:13:19.365827 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260620 08:13:19.366930 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:19.370460 645 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:19.370975 645 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:19.375982 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:19.368533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42951 (local address 127.0.0.1:45049)
0620 08:13:19.368572 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:19.368574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:19.368629 (+ 55us) server_negotiation.cc:407] Connection header received
0620 08:13:19.368742 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:19.368743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:19.368754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:19.368788 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:19.369497 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:19.370239 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:19.371287 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:19.371610 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:19.371655 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:19.373163 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:19.373165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:19.373165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:19.373172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:19.374692 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:19.375115 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:19.375117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:19.375117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:19.375169 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:19.375414 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:19.375415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:19.375415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:19.375532 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:19.375606 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:19.375828 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 08:13:19.375895 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:13:19.384191 645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddda055968484049936d99078673ecf1: Generated new TSK 0
I20260620 08:13:19.384369 645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:19.388662 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 563
W20260620 08:13:19.393199 548 connection.cc:535] client connection to 127.0.0.1:45049 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45049): Connection reset by peer (error 104)
2026-06-20T08:13:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[662](info): set up 2 sockets
Jun 20 08:13:19 dist-test-slave-z0sp krb5kdc[662](info): commencing operation
krb5kdc: starting...
W20260620 08:13:20.436951 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:20 dist-test-slave-z0sp krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:20.563200 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:20.642954 675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:20.643208 675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:20.643246 675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:20.644371 675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:20.644439 675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:20.644464 675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:20.644511 675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:20.644536 675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:20.646005 675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:20.646272 675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:20.646489 675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:20.651829 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:20.651616 681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:20.651649 680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:20.652554 675 server_base.cc:1034] running on GCE node
I20260620 08:13:20.652940 675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:20.653313 675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:20.654517 675 hybrid_clock.cc:648] HybridClock initialized: now 1781943200654478 us; error 39 us; skew 500 ppm
Jun 20 08:13:20 dist-test-slave-z0sp krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:20.657378 675 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:20.658082 675 webserver.cc:460] Webserver started at http://127.0.0.1:42627/ using document root <none> and password file <none>
I20260620 08:13:20.658385 675 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:20.658456 675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:20.658601 675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:20.659651 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "189543801ec148a3baf830d1e541c95c"
format_stamp: "Formatted at 2026-06-20 08:13:20 on dist-test-slave-z0sp"
server_key: "d8eb82857447e0261ca80d4d17d61c46"
server_key_iv: "55929a66844e135ab99f7dc36e271794"
server_key_version: "encryptionkey@0"
I20260620 08:13:20.660022 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "189543801ec148a3baf830d1e541c95c"
format_stamp: "Formatted at 2026-06-20 08:13:20 on dist-test-slave-z0sp"
server_key: "d8eb82857447e0261ca80d4d17d61c46"
server_key_iv: "55929a66844e135ab99f7dc36e271794"
server_key_version: "encryptionkey@0"
I20260620 08:13:20.661594 675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:20.662385 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:20.662592 675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:20.662704 675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "189543801ec148a3baf830d1e541c95c"
format_stamp: "Formatted at 2026-06-20 08:13:20 on dist-test-slave-z0sp"
server_key: "d8eb82857447e0261ca80d4d17d61c46"
server_key_iv: "55929a66844e135ab99f7dc36e271794"
server_key_version: "encryptionkey@0"
I20260620 08:13:20.662786 675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:20.675278 675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:20.677320 675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:20.677471 675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:20.682041 675 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41817
I20260620 08:13:20.682086 742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41817 every 8 connection(s)
I20260620 08:13:20.682435 675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:20.683254 743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:20.685524 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: Bootstrap starting.
I20260620 08:13:20.686090 743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:20.686383 743 log.cc:824] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:20.687119 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: No bootstrap required, opened a new log
I20260620 08:13:20.688465 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 675
I20260620 08:13:20.688633 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:20.688849 12819 external_mini_cluster.cc:1428] Setting key f2c1a8af5e6dca0c368227673dfc366c
I20260620 08:13:20.689230 743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260620 08:13:20.689458 743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:20.689523 743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 189543801ec148a3baf830d1e541c95c, State: Initialized, Role: FOLLOWER
I20260620 08:13:20.689680 743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260620 08:13:20.689764 743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:20.689795 743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:20.689841 743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:20.690992 743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260620 08:13:20.691099 743 leader_election.cc:302] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 189543801ec148a3baf830d1e541c95c; no voters:
I20260620 08:13:20.691247 743 leader_election.cc:288] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:20.691365 748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:20.691488 743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:20.691700 748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [term 1 LEADER]: Becoming Leader. State: Replica: 189543801ec148a3baf830d1e541c95c, State: Running, Role: LEADER
I20260620 08:13:20.691896 748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260620 08:13:20.692970 750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 189543801ec148a3baf830d1e541c95c. Latest consensus state: current_term: 1 leader_uuid: "189543801ec148a3baf830d1e541c95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } } }
I20260620 08:13:20.693248 750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:20.692965 749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "189543801ec148a3baf830d1e541c95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189543801ec148a3baf830d1e541c95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } } }
I20260620 08:13:20.693521 749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:20.693580 762 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:20.693652 762 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:20.694007 763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:20.694379 763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:20.695384 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:20.689389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47313 (local address 127.0.0.1:41817)
0620 08:13:20.689617 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:20.689626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:20.689658 (+ 32us) server_negotiation.cc:407] Connection header received
0620 08:13:20.689777 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:20.689784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:20.689838 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:20.689977 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:20.690187 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:20.691634 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:20.692880 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:20.693589 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:20.693720 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:20.694715 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:20.694719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:20.694719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:20.694924 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:20.694959 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:20.694974 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:20.695059 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:20.695150 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 08:13:20.695176 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":106,"threads_started":1}
I20260620 08:13:20.696028 763 catalog_manager.cc:1347] Generated new cluster ID: 82e1b74f6b9946d79800969d4e041fef
I20260620 08:13:20.696076 763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:20.696350 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:20.701257 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:20.697919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42319 (local address 127.0.0.1:41817)
0620 08:13:20.697953 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:20.697955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:20.698007 (+ 52us) server_negotiation.cc:407] Connection header received
0620 08:13:20.698057 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:20.698057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:20.698066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:20.698136 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:20.698237 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:20.698943 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:20.699884 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:20.700198 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:20.700276 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:20.701004 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:20.701005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:20.701005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:20.701010 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:20.701023 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:20.701028 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:20.701093 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:20.701165 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 08:13:20.701177 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:20.715230 763 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:20.715966 763 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:20.719696 763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 189543801ec148a3baf830d1e541c95c: Generated new TSK 0
I20260620 08:13:20.719944 763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:20.726897 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 675
W20260620 08:13:20.731741 660 connection.cc:535] client connection to 127.0.0.1:41817 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41817): Connection reset by peer (error 104)
2026-06-20T08:13:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:20 dist-test-slave-z0sp krb5kdc[774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:20 dist-test-slave-z0sp krb5kdc[774](info): set up 2 sockets
Jun 20 08:13:20 dist-test-slave-z0sp krb5kdc[774](info): commencing operation
krb5kdc: starting...
W20260620 08:13:21.773612 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:21 dist-test-slave-z0sp krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:21.897815 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45769
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:21.981067 787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:21.981272 787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:21.981297 787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:21.982488 787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:21.982528 787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:21.982543 787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:21.982556 787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:21.982600 787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:21.984017 787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45769
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:21.984257 787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:21.984469 787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:21.989199 795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:21.989212 792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:21.989259 787 server_base.cc:1034] running on GCE node
W20260620 08:13:21.989212 793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:21.989692 787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:21.989924 787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:21.991075 787 hybrid_clock.cc:648] HybridClock initialized: now 1781943201991063 us; error 34 us; skew 500 ppm
Jun 20 08:13:21 dist-test-slave-z0sp krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:22.000023 787 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:22.000479 787 webserver.cc:460] Webserver started at http://127.0.0.1:46471/ using document root <none> and password file <none>
I20260620 08:13:22.000758 787 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:22.000824 787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:22.000959 787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:22.002010 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "c27e3f8e823049f8b24caef831848c97"
format_stamp: "Formatted at 2026-06-20 08:13:21 on dist-test-slave-z0sp"
server_key: "64a724269e2f02969aa800a6a9549027"
server_key_iv: "46a53a9c886b0cdb3e2eb652ca8182a1"
server_key_version: "encryptionkey@0"
I20260620 08:13:22.002403 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "c27e3f8e823049f8b24caef831848c97"
format_stamp: "Formatted at 2026-06-20 08:13:21 on dist-test-slave-z0sp"
server_key: "64a724269e2f02969aa800a6a9549027"
server_key_iv: "46a53a9c886b0cdb3e2eb652ca8182a1"
server_key_version: "encryptionkey@0"
I20260620 08:13:22.003852 787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:22.004971 802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:22.005265 787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:22.005373 787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "c27e3f8e823049f8b24caef831848c97"
format_stamp: "Formatted at 2026-06-20 08:13:21 on dist-test-slave-z0sp"
server_key: "64a724269e2f02969aa800a6a9549027"
server_key_iv: "46a53a9c886b0cdb3e2eb652ca8182a1"
server_key_version: "encryptionkey@0"
I20260620 08:13:22.005431 787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:22.008584 787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:22.011076 787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:22.011264 787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:22.016177 787 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45769
I20260620 08:13:22.016196 854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45769 every 8 connection(s)
I20260620 08:13:22.016716 787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:22.017694 855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:22.019948 855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97: Bootstrap starting.
I20260620 08:13:22.020574 855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:22.020876 855 log.cc:824] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:22.021716 855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97: No bootstrap required, opened a new log
I20260620 08:13:22.022959 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 787
I20260620 08:13:22.023074 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:22.023339 12819 external_mini_cluster.cc:1428] Setting key 4e8d0e0cb40528bcb0822a8c837eba0d
I20260620 08:13:22.023586 855 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } }
I20260620 08:13:22.023784 855 raft_consensus.cc:374] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:22.023830 855 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27e3f8e823049f8b24caef831848c97, State: Initialized, Role: FOLLOWER
I20260620 08:13:22.023990 855 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } }
I20260620 08:13:22.024098 855 raft_consensus.cc:388] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:22.024159 855 raft_consensus.cc:482] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:22.024217 855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:22.025336 855 raft_consensus.cc:504] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } }
I20260620 08:13:22.025450 855 leader_election.cc:302] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c27e3f8e823049f8b24caef831848c97; no voters:
I20260620 08:13:22.025617 855 leader_election.cc:288] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:22.025661 860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:22.025866 860 raft_consensus.cc:686] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [term 1 LEADER]: Becoming Leader. State: Replica: c27e3f8e823049f8b24caef831848c97, State: Running, Role: LEADER
I20260620 08:13:22.025861 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:22.023996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60815 (local address 127.0.0.1:45769)
0620 08:13:22.024243 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:22.024253 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:22.024298 (+ 45us) server_negotiation.cc:407] Connection header received
0620 08:13:22.024420 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:22.024426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:22.024454 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:22.024625 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:22.025226 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:22.025229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:22.025230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:22.025235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:22.025291 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:22.025306 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:22.025384 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:22.025422 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 08:13:22.025557 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260620 08:13:22.025971 855 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:22.026019 860 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } }
I20260620 08:13:22.026489 861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c27e3f8e823049f8b24caef831848c97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } } }
I20260620 08:13:22.026500 862 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27e3f8e823049f8b24caef831848c97. Latest consensus state: current_term: 1 leader_uuid: "c27e3f8e823049f8b24caef831848c97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27e3f8e823049f8b24caef831848c97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45769 } } }
I20260620 08:13:22.026578 861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:22.026583 862 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:22.027217 867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:22.027735 867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:22.028054 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:22.029256 867 catalog_manager.cc:1347] Generated new cluster ID: c4b83a186289447e9c875ed1ee0d9b9a
I20260620 08:13:22.029307 867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:22.033979 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:22.032889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41359 (local address 127.0.0.1:45769)
0620 08:13:22.032926 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:22.032927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:22.033059 (+ 132us) server_negotiation.cc:407] Connection header received
0620 08:13:22.033119 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:22.033119 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:22.033125 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:22.033174 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:22.033736 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:22.033737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:22.033738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:22.033742 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:22.033755 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:22.033759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:22.033806 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:22.033869 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 08:13:22.033884 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:13:22.048529 867 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:22.048708 867 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:22.059655 867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c27e3f8e823049f8b24caef831848c97: Generated new TSK 0
I20260620 08:13:22.059836 867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:22.063262 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 787
2026-06-20T08:13:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:22 dist-test-slave-z0sp krb5kdc[886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:22 dist-test-slave-z0sp krb5kdc[886](info): set up 2 sockets
Jun 20 08:13:22 dist-test-slave-z0sp krb5kdc[886](info): commencing operation
krb5kdc: starting...
W20260620 08:13:23.133121 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:23.255735 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:23.342876 899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:23.343122 899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:23.343158 899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:23.344321 899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:23.344384 899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:23.344409 899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:23.344430 899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:23.344456 899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:23.345979 899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:23.346253 899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:23.346485 899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:23.351328 904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:23.351420 907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:23.351576 905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:23.351742 899 server_base.cc:1034] running on GCE node
I20260620 08:13:23.351985 899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:23.352267 899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:23.353466 899 hybrid_clock.cc:648] HybridClock initialized: now 1781943203353438 us; error 50 us; skew 500 ppm
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:23.356398 899 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:23.357151 899 webserver.cc:460] Webserver started at http://127.0.0.1:33301/ using document root <none> and password file <none>
I20260620 08:13:23.357455 899 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:23.357533 899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:23.357697 899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:23.358963 899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "d08982f9172f4d71830369fdf999e2e4"
format_stamp: "Formatted at 2026-06-20 08:13:23 on dist-test-slave-z0sp"
server_key: "8c03557f9e608b3b4fc711dea486ab1e"
server_key_iv: "0c8b0b8b1ba2fd303727018b4a3fa8d1"
server_key_version: "encryptionkey@0"
I20260620 08:13:23.359375 899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "d08982f9172f4d71830369fdf999e2e4"
format_stamp: "Formatted at 2026-06-20 08:13:23 on dist-test-slave-z0sp"
server_key: "8c03557f9e608b3b4fc711dea486ab1e"
server_key_iv: "0c8b0b8b1ba2fd303727018b4a3fa8d1"
server_key_version: "encryptionkey@0"
I20260620 08:13:23.361128 899 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260620 08:13:23.362196 914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:23.362430 899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:23.362538 899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "d08982f9172f4d71830369fdf999e2e4"
format_stamp: "Formatted at 2026-06-20 08:13:23 on dist-test-slave-z0sp"
server_key: "8c03557f9e608b3b4fc711dea486ab1e"
server_key_iv: "0c8b0b8b1ba2fd303727018b4a3fa8d1"
server_key_version: "encryptionkey@0"
I20260620 08:13:23.362638 899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:23.369664 899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:23.371830 899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:23.371982 899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:23.376011 899 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44271
I20260620 08:13:23.376044 966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44271 every 8 connection(s)
I20260620 08:13:23.376399 899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:23.377279 967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:23.379467 967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4: Bootstrap starting.
I20260620 08:13:23.380024 967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:23.380309 967 log.cc:824] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:23.381266 967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4: No bootstrap required, opened a new log
I20260620 08:13:23.381860 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 899
I20260620 08:13:23.382006 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:23.382181 12819 external_mini_cluster.cc:1428] Setting key a6297f55b44aa11165ed3bf48eac8134
I20260620 08:13:23.383481 967 raft_consensus.cc:348] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } }
I20260620 08:13:23.383662 967 raft_consensus.cc:374] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:23.383702 967 raft_consensus.cc:729] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08982f9172f4d71830369fdf999e2e4, State: Initialized, Role: FOLLOWER
I20260620 08:13:23.383827 967 consensus_queue.cc:260] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } }
I20260620 08:13:23.383916 967 raft_consensus.cc:388] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:23.383940 967 raft_consensus.cc:482] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:23.383970 967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:23.385001 967 raft_consensus.cc:504] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } }
I20260620 08:13:23.385110 967 leader_election.cc:302] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d08982f9172f4d71830369fdf999e2e4; no voters:
I20260620 08:13:23.385253 967 leader_election.cc:288] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:23.385321 972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:23.385483 972 raft_consensus.cc:686] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [term 1 LEADER]: Becoming Leader. State: Replica: d08982f9172f4d71830369fdf999e2e4, State: Running, Role: LEADER
I20260620 08:13:23.385601 972 consensus_queue.cc:237] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } }
I20260620 08:13:23.385476 967 sys_catalog.cc:564] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:23.386004 974 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d08982f9172f4d71830369fdf999e2e4. Latest consensus state: current_term: 1 leader_uuid: "d08982f9172f4d71830369fdf999e2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } } }
I20260620 08:13:23.386150 974 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:23.386345 973 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d08982f9172f4d71830369fdf999e2e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08982f9172f4d71830369fdf999e2e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44271 } } }
I20260620 08:13:23.386430 973 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:23.386560 976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:23.387056 976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:23.388736 976 catalog_manager.cc:1347] Generated new cluster ID: 0e5d7c4b71274fbcb925f0d638bcb3e8
I20260620 08:13:23.388792 976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 08:13:23.394694 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:23.382826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34499 (local address 127.0.0.1:44271)
0620 08:13:23.383072 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:23.383082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:23.383270 (+ 188us) server_negotiation.cc:407] Connection header received
0620 08:13:23.383445 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:23.383453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:23.383509 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:23.383669 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:23.384733 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:23.386078 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:23.387339 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:23.387953 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:23.388068 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:23.390977 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:23.390981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:23.390981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:23.390994 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:23.393031 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:23.393411 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:23.393414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:23.393415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:23.393533 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:23.393731 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:23.393732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:23.393732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:23.394110 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:23.394279 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:23.394348 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 08:13:23.394488 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":125,"threads_started":1}
I20260620 08:13:23.395633 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:23.402423 976 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:23.403095 976 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:23.404270 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:23.397201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45173 (local address 127.0.0.1:44271)
0620 08:13:23.397244 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:23.397245 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:23.397355 (+ 110us) server_negotiation.cc:407] Connection header received
0620 08:13:23.397449 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:23.397450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:23.397460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:23.397489 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:23.398191 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:23.398915 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:23.399954 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:23.400278 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:23.400318 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:23.401731 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:23.401732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:23.401732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:23.401745 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:23.403190 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:23.403518 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:23.403519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:23.403519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:23.403569 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:23.403782 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:23.403783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:23.403783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:23.403886 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:23.403983 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:23.404082 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 08:13:23.404170 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 08:13:23.411029 976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d08982f9172f4d71830369fdf999e2e4: Generated new TSK 0
I20260620 08:13:23.411203 976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:23.413270 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 899
2026-06-20T08:13:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[998](info): set up 2 sockets
Jun 20 08:13:23 dist-test-slave-z0sp krb5kdc[998](info): commencing operation
krb5kdc: starting...
W20260620 08:13:24.453805 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:24 dist-test-slave-z0sp krb5kdc[998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:24.575500 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:24.659914 1011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:24.660105 1011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:24.660128 1011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:24.661258 1011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:24.661300 1011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:24.661314 1011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:24.661327 1011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:24.661339 1011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:24.662791 1011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:24.663003 1011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:24.663193 1011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:24.668542 1017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:24.668599 1016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:24.668625 1019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:24.668859 1011 server_base.cc:1034] running on GCE node
I20260620 08:13:24.669194 1011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:24.669494 1011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:24.670682 1011 hybrid_clock.cc:648] HybridClock initialized: now 1781943204670655 us; error 40 us; skew 500 ppm
Jun 20 08:13:24 dist-test-slave-z0sp krb5kdc[998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:24.673458 1011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:24.673935 1011 webserver.cc:460] Webserver started at http://127.0.0.1:35305/ using document root <none> and password file <none>
I20260620 08:13:24.674151 1011 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:24.674213 1011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:24.674356 1011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:24.675412 1011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "0359defe71f64bd7a05f336b70ef89f7"
format_stamp: "Formatted at 2026-06-20 08:13:24 on dist-test-slave-z0sp"
server_key: "13c667a1f32d261f757ae80c037025f0"
server_key_iv: "961454ff3c9911a149d7df399536c1c8"
server_key_version: "encryptionkey@0"
I20260620 08:13:24.675768 1011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "0359defe71f64bd7a05f336b70ef89f7"
format_stamp: "Formatted at 2026-06-20 08:13:24 on dist-test-slave-z0sp"
server_key: "13c667a1f32d261f757ae80c037025f0"
server_key_iv: "961454ff3c9911a149d7df399536c1c8"
server_key_version: "encryptionkey@0"
I20260620 08:13:24.677171 1011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 08:13:24.677984 1026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:24.678177 1011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:24.678277 1011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "0359defe71f64bd7a05f336b70ef89f7"
format_stamp: "Formatted at 2026-06-20 08:13:24 on dist-test-slave-z0sp"
server_key: "13c667a1f32d261f757ae80c037025f0"
server_key_iv: "961454ff3c9911a149d7df399536c1c8"
server_key_version: "encryptionkey@0"
I20260620 08:13:24.678345 1011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:24.698913 1011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:24.702086 1011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:24.702284 1011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:24.706656 1011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35133
I20260620 08:13:24.706712 1078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35133 every 8 connection(s)
I20260620 08:13:24.707072 1011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:24.707916 1079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:24.710165 1079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: Bootstrap starting.
I20260620 08:13:24.710569 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1011
I20260620 08:13:24.710757 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:24.710807 1079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:24.711022 12819 external_mini_cluster.cc:1428] Setting key 39ec4d8bd9070c355f50c226295a0fda
I20260620 08:13:24.711163 1079 log.cc:824] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:24.712004 1079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: No bootstrap required, opened a new log
I20260620 08:13:24.714583 1079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260620 08:13:24.714798 1079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:24.714824 1079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0359defe71f64bd7a05f336b70ef89f7, State: Initialized, Role: FOLLOWER
I20260620 08:13:24.715029 1079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260620 08:13:24.715121 1079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:24.715149 1079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:24.715220 1079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:24.716329 1079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260620 08:13:24.716435 1079 leader_election.cc:302] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0359defe71f64bd7a05f336b70ef89f7; no voters:
I20260620 08:13:24.716773 1079 leader_election.cc:288] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:24.716971 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:24.711727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34245 (local address 127.0.0.1:35133)
0620 08:13:24.711915 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:24.711922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:24.711947 (+ 25us) server_negotiation.cc:407] Connection header received
0620 08:13:24.712412 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:24.712419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:24.712464 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:24.712641 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:24.712878 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:24.713791 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:24.714567 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:24.715155 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:24.715277 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:24.716210 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:24.716214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:24.716214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:24.716382 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:24.716449 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:24.716464 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:24.716582 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:24.716666 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 08:13:24.716730 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260620 08:13:24.717062 1085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:24.717131 1079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:24.718092 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:24.718562 1085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [term 1 LEADER]: Becoming Leader. State: Replica: 0359defe71f64bd7a05f336b70ef89f7, State: Running, Role: LEADER
I20260620 08:13:24.718695 1085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260620 08:13:24.719273 1084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0359defe71f64bd7a05f336b70ef89f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260620 08:13:24.719367 1084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:24.719754 1096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0359defe71f64bd7a05f336b70ef89f7. Latest consensus state: current_term: 1 leader_uuid: "0359defe71f64bd7a05f336b70ef89f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0359defe71f64bd7a05f336b70ef89f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260620 08:13:24.719846 1096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7 [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:24.720319 1105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:24.720382 1105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:24.720435 1102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:24.720621 1102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:24.722128 1102 catalog_manager.cc:1347] Generated new cluster ID: e9ea46f1420843a98b676a18155955fa
I20260620 08:13:24.722188 1102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:24.723237 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:24.719662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45699 (local address 127.0.0.1:35133)
0620 08:13:24.719711 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:24.719713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:24.720053 (+ 340us) server_negotiation.cc:407] Connection header received
0620 08:13:24.720140 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:24.720141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:24.720156 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:24.720209 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:24.720361 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:24.721082 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:24.721970 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:24.722250 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:24.722296 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:24.723013 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:24.723014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:24.723014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:24.723018 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:24.723030 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:24.723034 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:24.723057 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:24.723147 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 08:13:24.723168 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:24.732219 1102 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:24.732816 1102 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:24.735984 1102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0359defe71f64bd7a05f336b70ef89f7: Generated new TSK 0
I20260620 08:13:24.736150 1102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:24.741309 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1011
2026-06-20T08:13:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:24 dist-test-slave-z0sp krb5kdc[1110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:24 dist-test-slave-z0sp krb5kdc[1110](info): set up 2 sockets
Jun 20 08:13:24 dist-test-slave-z0sp krb5kdc[1110](info): commencing operation
krb5kdc: starting...
W20260620 08:13:25.781038 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:25 dist-test-slave-z0sp krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:25.913678 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:25.998167 1123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:25.998373 1123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:25.998397 1123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:25.999523 1123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:25.999570 1123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:25.999584 1123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:25.999598 1123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:25.999609 1123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:26.001154 1123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:26.001345 1123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:26.001573 1123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:26.006613 1129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:26.006613 1128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:26.007014 1123 server_base.cc:1034] running on GCE node
W20260620 08:13:26.007517 1131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:26.007781 1123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:26.008052 1123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:26.009225 1123 hybrid_clock.cc:648] HybridClock initialized: now 1781943206009213 us; error 28 us; skew 500 ppm
Jun 20 08:13:26 dist-test-slave-z0sp krb5kdc[1110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:26.011992 1123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:26.012449 1123 webserver.cc:460] Webserver started at http://127.0.0.1:34567/ using document root <none> and password file <none>
I20260620 08:13:26.012704 1123 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:26.012753 1123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:26.012876 1123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:26.013926 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "62339b0832374d8c8e79fdc91de7b50e"
format_stamp: "Formatted at 2026-06-20 08:13:26 on dist-test-slave-z0sp"
server_key: "12bb35cc5ddc5e4bbfe9dd775430f125"
server_key_iv: "d673c1c77208df74222549cf90ac391a"
server_key_version: "encryptionkey@0"
I20260620 08:13:26.014281 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "62339b0832374d8c8e79fdc91de7b50e"
format_stamp: "Formatted at 2026-06-20 08:13:26 on dist-test-slave-z0sp"
server_key: "12bb35cc5ddc5e4bbfe9dd775430f125"
server_key_iv: "d673c1c77208df74222549cf90ac391a"
server_key_version: "encryptionkey@0"
I20260620 08:13:26.015848 1123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:26.017100 1138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:26.017377 1123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:26.017479 1123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "62339b0832374d8c8e79fdc91de7b50e"
format_stamp: "Formatted at 2026-06-20 08:13:26 on dist-test-slave-z0sp"
server_key: "12bb35cc5ddc5e4bbfe9dd775430f125"
server_key_iv: "d673c1c77208df74222549cf90ac391a"
server_key_version: "encryptionkey@0"
I20260620 08:13:26.017532 1123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:26.020720 1123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:26.022891 1123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:26.023000 1123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:26.026645 1123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38475
I20260620 08:13:26.026731 1190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38475 every 8 connection(s)
I20260620 08:13:26.027019 1123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:26.027869 1191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:26.028764 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1123
I20260620 08:13:26.028887 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:26.029088 12819 external_mini_cluster.cc:1428] Setting key 38911fe677f6746195c3f75d7e1adb0f
I20260620 08:13:26.030900 1191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e: Bootstrap starting.
I20260620 08:13:26.031615 1191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:26.031944 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:26.029638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40035 (local address 127.0.0.1:38475)
0620 08:13:26.029805 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:26.029814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:26.029877 (+ 63us) server_negotiation.cc:407] Connection header received
0620 08:13:26.030591 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:26.030599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:26.030633 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:26.030809 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:26.031491 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:26.031495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:26.031495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:26.031499 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:26.031595 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:26.031609 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:26.031717 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:26.031767 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 08:13:26.031809 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260620 08:13:26.031978 1191 log.cc:824] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:26.032840 1191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e: No bootstrap required, opened a new log
I20260620 08:13:26.033026 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:26.034976 1191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260620 08:13:26.035128 1191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:26.035213 1191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62339b0832374d8c8e79fdc91de7b50e, State: Initialized, Role: FOLLOWER
I20260620 08:13:26.035357 1191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260620 08:13:26.035468 1191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:26.035516 1191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:26.035605 1191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:26.035847 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:26.034394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38971 (local address 127.0.0.1:38475)
0620 08:13:26.034436 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:26.034438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:26.034495 (+ 57us) server_negotiation.cc:407] Connection header received
0620 08:13:26.034572 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:26.034573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:26.034577 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:26.034644 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:26.035466 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:26.035468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:26.035468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:26.035476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:26.035499 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:26.035505 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:26.035571 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:26.035683 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 08:13:26.035702 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 08:13:26.036352 1191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260620 08:13:26.036516 1191 leader_election.cc:302] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62339b0832374d8c8e79fdc91de7b50e; no voters:
I20260620 08:13:26.036682 1191 leader_election.cc:288] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:26.036779 1201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:26.036953 1191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:26.037000 1201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [term 1 LEADER]: Becoming Leader. State: Replica: 62339b0832374d8c8e79fdc91de7b50e, State: Running, Role: LEADER
I20260620 08:13:26.037209 1201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } }
I20260620 08:13:26.037746 1203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62339b0832374d8c8e79fdc91de7b50e. Latest consensus state: current_term: 1 leader_uuid: "62339b0832374d8c8e79fdc91de7b50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } } }
I20260620 08:13:26.037734 1202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62339b0832374d8c8e79fdc91de7b50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62339b0832374d8c8e79fdc91de7b50e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38475 } } }
I20260620 08:13:26.037982 1203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:26.037982 1202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:26.038451 1213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:26.039285 1213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:26.040968 1213 catalog_manager.cc:1347] Generated new cluster ID: 868e26079c554bffa54fc453f3ac2135
I20260620 08:13:26.041024 1213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:26.056434 1213 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:26.056638 1213 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:26.063824 1213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62339b0832374d8c8e79fdc91de7b50e: Generated new TSK 0
I20260620 08:13:26.064152 1213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:26.070644 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1123
2026-06-20T08:13:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:26 dist-test-slave-z0sp krb5kdc[1222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:26 dist-test-slave-z0sp krb5kdc[1222](info): set up 2 sockets
Jun 20 08:13:26 dist-test-slave-z0sp krb5kdc[1222](info): commencing operation
krb5kdc: starting...
W20260620 08:13:27.125049 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:27.248705 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:27.337224 1235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:27.337443 1235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:27.337471 1235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:27.338717 1235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:27.338763 1235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:27.338778 1235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:27.338790 1235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:27.338802 1235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:27.340284 1235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:27.340543 1235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:27.340762 1235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:27.345894 1241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:27.345834 1240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:27.345822 1243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:27.346302 1235 server_base.cc:1034] running on GCE node
I20260620 08:13:27.346626 1235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:27.347033 1235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:27.348225 1235 hybrid_clock.cc:648] HybridClock initialized: now 1781943207348232 us; error 59 us; skew 500 ppm
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:27.356094 1235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:27.356732 1235 webserver.cc:460] Webserver started at http://127.0.0.1:45379/ using document root <none> and password file <none>
I20260620 08:13:27.357018 1235 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:27.357075 1235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:27.357268 1235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:27.358619 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06"
format_stamp: "Formatted at 2026-06-20 08:13:27 on dist-test-slave-z0sp"
server_key: "2591ba8c536fbdb486442bac3e930853"
server_key_iv: "12bfa5cd504d2e9a8a0da2b58a915315"
server_key_version: "encryptionkey@0"
I20260620 08:13:27.359079 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06"
format_stamp: "Formatted at 2026-06-20 08:13:27 on dist-test-slave-z0sp"
server_key: "2591ba8c536fbdb486442bac3e930853"
server_key_iv: "12bfa5cd504d2e9a8a0da2b58a915315"
server_key_version: "encryptionkey@0"
I20260620 08:13:27.360864 1235 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 08:13:27.361789 1250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:27.361999 1235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:27.362092 1235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06"
format_stamp: "Formatted at 2026-06-20 08:13:27 on dist-test-slave-z0sp"
server_key: "2591ba8c536fbdb486442bac3e930853"
server_key_iv: "12bfa5cd504d2e9a8a0da2b58a915315"
server_key_version: "encryptionkey@0"
I20260620 08:13:27.362258 1235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:27.387678 1235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:27.389940 1235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:27.390115 1235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:27.394273 1235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36499
I20260620 08:13:27.394326 1302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36499 every 8 connection(s)
I20260620 08:13:27.394694 1235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:27.395908 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:27.399019 1303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06: Bootstrap starting.
I20260620 08:13:27.399710 1303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:27.400010 1303 log.cc:824] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:27.400785 1303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06: No bootstrap required, opened a new log
I20260620 08:13:27.402645 1303 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260620 08:13:27.402786 1303 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:27.402809 1303 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1de6f1cdd41427aa9ed0e57c8cedc06, State: Initialized, Role: FOLLOWER
I20260620 08:13:27.402968 1303 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260620 08:13:27.403033 1303 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:27.403054 1303 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:27.403102 1303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:27.403780 1303 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260620 08:13:27.403896 1303 leader_election.cc:302] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1de6f1cdd41427aa9ed0e57c8cedc06; no voters:
I20260620 08:13:27.404057 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1235
I20260620 08:13:27.404093 1303 leader_election.cc:288] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:27.404116 1306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:27.404213 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:27.404228 1306 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [term 1 LEADER]: Becoming Leader. State: Replica: c1de6f1cdd41427aa9ed0e57c8cedc06, State: Running, Role: LEADER
I20260620 08:13:27.404302 1306 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } }
I20260620 08:13:27.404457 12819 external_mini_cluster.cc:1428] Setting key 0fbb90a67945979eac6e018614b92279
I20260620 08:13:27.404356 1303 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:27.404970 1307 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } } }
I20260620 08:13:27.405170 1307 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:27.405922 1316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:27.406085 1308 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1de6f1cdd41427aa9ed0e57c8cedc06. Latest consensus state: current_term: 1 leader_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1de6f1cdd41427aa9ed0e57c8cedc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36499 } } }
I20260620 08:13:27.406318 1308 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:27.406464 1316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:27.407963 1316 catalog_manager.cc:1347] Generated new cluster ID: 021b81a84d1142bf9736d0b4e629d2bc
I20260620 08:13:27.408013 1316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 08:13:27.417436 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:27.405348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:36893 (local address 127.0.0.1:36499)
0620 08:13:27.405539 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:27.405624 (+ 85us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:27.406039 (+ 415us) server_negotiation.cc:407] Connection header received
0620 08:13:27.406381 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:27.406390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:27.406459 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:27.406581 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:27.407395 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:27.408793 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:27.409819 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:27.410472 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:27.410584 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:27.413719 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:27.413724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:27.413724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:27.413737 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:27.415734 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:27.416193 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:27.416195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:27.416196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:27.416293 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:27.416551 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:27.416552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:27.416552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:27.416851 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:27.417017 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:27.417128 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 08:13:27.417219 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260620 08:13:27.418427 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:27.424639 1316 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:27.425335 1316 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:27.426841 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:27.419822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:55791 (local address 127.0.0.1:36499)
0620 08:13:27.419856 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:27.419858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:27.419900 (+ 42us) server_negotiation.cc:407] Connection header received
0620 08:13:27.419956 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:27.419957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:27.419965 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:27.420029 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:27.420599 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:27.421390 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:27.422340 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:27.422668 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:27.422738 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:27.424013 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:27.424014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:27.424015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:27.424022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:27.425609 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:27.426037 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:27.426038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:27.426039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:27.426092 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:27.426329 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:27.426330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:27.426330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:27.426444 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:27.426542 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:27.426676 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 08:13:27.426729 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:13:27.433230 1316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1de6f1cdd41427aa9ed0e57c8cedc06: Generated new TSK 0
I20260620 08:13:27.433418 1316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:27.439193 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1235
W20260620 08:13:27.443783 1220 connection.cc:535] client connection to 127.0.0.1:36499 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36499): Connection reset by peer (error 104)
2026-06-20T08:13:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1334](info): set up 2 sockets
Jun 20 08:13:27 dist-test-slave-z0sp krb5kdc[1334](info): commencing operation
krb5kdc: starting...
W20260620 08:13:28.530328 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:28 dist-test-slave-z0sp krb5kdc[1334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:28.696190 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:28.780540 1347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:28.780776 1347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:28.780799 1347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:28.782073 1347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:28.782120 1347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:28.782135 1347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:28.782147 1347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:28.782159 1347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:28.785295 1347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:28.785699 1347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:28.786103 1347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:28.791996 1355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:28.792151 1352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:28.792227 1347 server_base.cc:1034] running on GCE node
W20260620 08:13:28.792130 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:28.792618 1347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:28.792963 1347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:28.794128 1347 hybrid_clock.cc:648] HybridClock initialized: now 1781943208794108 us; error 38 us; skew 500 ppm
Jun 20 08:13:28 dist-test-slave-z0sp krb5kdc[1334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:28.797322 1347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:28.797856 1347 webserver.cc:460] Webserver started at http://127.0.0.1:39141/ using document root <none> and password file <none>
I20260620 08:13:28.798112 1347 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:28.798179 1347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:28.798328 1347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:28.799404 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "594b2c64e20b4f7794d33f828411b806"
format_stamp: "Formatted at 2026-06-20 08:13:28 on dist-test-slave-z0sp"
server_key: "35c7ffbec445655c94ad0e2ef5c3da5c"
server_key_iv: "dc4039edb6a7da1218b520a0343f1e2c"
server_key_version: "encryptionkey@0"
I20260620 08:13:28.799774 1347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "594b2c64e20b4f7794d33f828411b806"
format_stamp: "Formatted at 2026-06-20 08:13:28 on dist-test-slave-z0sp"
server_key: "35c7ffbec445655c94ad0e2ef5c3da5c"
server_key_iv: "dc4039edb6a7da1218b520a0343f1e2c"
server_key_version: "encryptionkey@0"
I20260620 08:13:28.801275 1347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:28.802314 1362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:28.802632 1347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:28.802772 1347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "594b2c64e20b4f7794d33f828411b806"
format_stamp: "Formatted at 2026-06-20 08:13:28 on dist-test-slave-z0sp"
server_key: "35c7ffbec445655c94ad0e2ef5c3da5c"
server_key_iv: "dc4039edb6a7da1218b520a0343f1e2c"
server_key_version: "encryptionkey@0"
I20260620 08:13:28.802853 1347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:28.821978 1347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:28.825423 1347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:28.825621 1347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:28.830516 1347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34599
I20260620 08:13:28.830556 1414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34599 every 8 connection(s)
I20260620 08:13:28.830936 1347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:28.832049 1415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:28.832283 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1347
I20260620 08:13:28.832470 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:28.832753 12819 external_mini_cluster.cc:1428] Setting key 1fedd594ee6f4f76be872404dfe9f076
I20260620 08:13:28.835291 1415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806: Bootstrap starting.
I20260620 08:13:28.835980 1415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:28.836285 1415 log.cc:824] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:28.836995 1415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806: No bootstrap required, opened a new log
I20260620 08:13:28.839530 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:28.833426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:49245 (local address 127.0.0.1:34599)
0620 08:13:28.833627 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:28.833637 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:28.833670 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:13:28.834199 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:28.834208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:28.834282 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:28.834431 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:28.834630 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:28.836023 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:28.836960 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:28.837568 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:28.837677 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:28.838678 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:28.838683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:28.838684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:28.838922 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:28.838972 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:28.838988 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:28.839110 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:28.839270 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 08:13:28.839298 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260620 08:13:28.839504 1415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } }
I20260620 08:13:28.839663 1415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:28.839695 1415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594b2c64e20b4f7794d33f828411b806, State: Initialized, Role: FOLLOWER
I20260620 08:13:28.839860 1415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } }
I20260620 08:13:28.839947 1415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:28.839972 1415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:28.840044 1415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:28.840679 1415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } }
I20260620 08:13:28.840806 1415 leader_election.cc:302] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 594b2c64e20b4f7794d33f828411b806; no voters:
I20260620 08:13:28.840978 1415 leader_election.cc:288] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:28.841065 1420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:28.841116 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:28.841248 1415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:28.841365 1420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [term 1 LEADER]: Becoming Leader. State: Replica: 594b2c64e20b4f7794d33f828411b806, State: Running, Role: LEADER
I20260620 08:13:28.841524 1420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } }
I20260620 08:13:28.842027 1421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "594b2c64e20b4f7794d33f828411b806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } } }
I20260620 08:13:28.842115 1421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:28.842314 1421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 594b2c64e20b4f7794d33f828411b806. Latest consensus state: current_term: 1 leader_uuid: "594b2c64e20b4f7794d33f828411b806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594b2c64e20b4f7794d33f828411b806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34599 } } }
I20260620 08:13:28.842414 1421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:28.843005 1429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:28.843668 1429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:28.845851 1429 catalog_manager.cc:1347] Generated new cluster ID: 4a6fbe1d85884131a02ca2e09433b627
I20260620 08:13:28.845918 1429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:28.847401 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:28.843677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:46251 (local address 127.0.0.1:34599)
0620 08:13:28.843721 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:28.843724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:28.843926 (+ 202us) server_negotiation.cc:407] Connection header received
0620 08:13:28.844032 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:28.844033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:28.844043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:28.844077 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:28.844288 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:28.844951 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:28.845770 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:28.846041 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:28.846115 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:28.847114 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:28.847115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:28.847115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:28.847119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:28.847135 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:28.847139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:28.847192 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:28.847318 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 08:13:28.847328 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:28.857023 1429 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:28.858075 1429 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:28.875000 1429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 594b2c64e20b4f7794d33f828411b806: Generated new TSK 0
I20260620 08:13:28.875221 1429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:28.884788 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1347
W20260620 08:13:28.889495 1332 connection.cc:535] client connection to 127.0.0.1:34599 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34599): Connection reset by peer (error 104)
2026-06-20T08:13:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:28 dist-test-slave-z0sp krb5kdc[1446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:28 dist-test-slave-z0sp krb5kdc[1446](info): set up 2 sockets
Jun 20 08:13:28 dist-test-slave-z0sp krb5kdc[1446](info): commencing operation
krb5kdc: starting...
W20260620 08:13:29.949891 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:30 dist-test-slave-z0sp krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:30.075824 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:30.163370 1459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:30.163576 1459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:30.163599 1459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:30.164672 1459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:30.164709 1459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:30.164723 1459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:30.164736 1459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:30.164748 1459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:30.166112 1459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:30.166290 1459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:30.166453 1459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:30.171422 1465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:30.171468 1467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:30.171418 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:30.171653 1459 server_base.cc:1034] running on GCE node
I20260620 08:13:30.171909 1459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:30.172179 1459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:30.173364 1459 hybrid_clock.cc:648] HybridClock initialized: now 1781943210173343 us; error 45 us; skew 500 ppm
Jun 20 08:13:30 dist-test-slave-z0sp krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:30.176194 1459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:30.176677 1459 webserver.cc:460] Webserver started at http://127.0.0.1:35579/ using document root <none> and password file <none>
I20260620 08:13:30.176904 1459 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:30.176973 1459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:30.177120 1459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:30.178164 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "61bde287fbfb46f3b2663592aaa8a077"
format_stamp: "Formatted at 2026-06-20 08:13:30 on dist-test-slave-z0sp"
server_key: "5c7b3079801df37429aebc6f4a296367"
server_key_iv: "0e6f26cf250e4cfcc84f9f1c2c8f80a8"
server_key_version: "encryptionkey@0"
I20260620 08:13:30.178534 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "61bde287fbfb46f3b2663592aaa8a077"
format_stamp: "Formatted at 2026-06-20 08:13:30 on dist-test-slave-z0sp"
server_key: "5c7b3079801df37429aebc6f4a296367"
server_key_iv: "0e6f26cf250e4cfcc84f9f1c2c8f80a8"
server_key_version: "encryptionkey@0"
I20260620 08:13:30.180044 1459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:30.180948 1474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:30.181181 1459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 08:13:30.181290 1459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "61bde287fbfb46f3b2663592aaa8a077"
format_stamp: "Formatted at 2026-06-20 08:13:30 on dist-test-slave-z0sp"
server_key: "5c7b3079801df37429aebc6f4a296367"
server_key_iv: "0e6f26cf250e4cfcc84f9f1c2c8f80a8"
server_key_version: "encryptionkey@0"
I20260620 08:13:30.181381 1459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:30.184571 1459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:30.186861 1459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:30.186981 1459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:30.190896 1459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40207
I20260620 08:13:30.190981 1526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40207 every 8 connection(s)
I20260620 08:13:30.191273 1459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:30.191637 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1459
I20260620 08:13:30.191752 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:30.191983 12819 external_mini_cluster.cc:1428] Setting key 76511a53aa37d95e038496456003494d
I20260620 08:13:30.192338 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:30.194317 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:30.192478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:45999 (local address 127.0.0.1:40207)
0620 08:13:30.192668 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:30.192674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:30.192698 (+ 24us) server_negotiation.cc:407] Connection header received
0620 08:13:30.193099 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:30.193104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:30.193127 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:30.193232 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:30.193881 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:30.193883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:30.193884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:30.193887 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:30.193924 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:30.193934 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:30.193989 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:30.194078 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 08:13:30.194133 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260620 08:13:30.195202 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:30.195595 1527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077: Bootstrap starting.
I20260620 08:13:30.196224 1527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:30.196627 1527 log.cc:824] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:30.197412 1527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077: No bootstrap required, opened a new log
I20260620 08:13:30.198225 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:30.196652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:37113 (local address 127.0.0.1:40207)
0620 08:13:30.196684 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:30.196686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:30.196947 (+ 261us) server_negotiation.cc:407] Connection header received
0620 08:13:30.196971 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:30.196972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:30.196977 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:30.197042 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:30.197887 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:30.197889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:30.197890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:30.197896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:30.197918 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:30.197924 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:30.197988 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:30.198109 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 08:13:30.198130 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:13:30.199573 1527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260620 08:13:30.199709 1527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:30.199733 1527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61bde287fbfb46f3b2663592aaa8a077, State: Initialized, Role: FOLLOWER
I20260620 08:13:30.199851 1527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260620 08:13:30.199934 1527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:30.199955 1527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:30.199982 1527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:30.200601 1527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260620 08:13:30.200687 1527 leader_election.cc:302] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61bde287fbfb46f3b2663592aaa8a077; no voters:
I20260620 08:13:30.200850 1527 leader_election.cc:288] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:30.200945 1537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:30.201125 1527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:30.201107 1537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [term 1 LEADER]: Becoming Leader. State: Replica: 61bde287fbfb46f3b2663592aaa8a077, State: Running, Role: LEADER
I20260620 08:13:30.201336 1537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } }
I20260620 08:13:30.201745 1538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61bde287fbfb46f3b2663592aaa8a077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260620 08:13:30.201769 1539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61bde287fbfb46f3b2663592aaa8a077. Latest consensus state: current_term: 1 leader_uuid: "61bde287fbfb46f3b2663592aaa8a077" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bde287fbfb46f3b2663592aaa8a077" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40207 } } }
I20260620 08:13:30.201838 1538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:30.201869 1539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:30.202034 1546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:30.202571 1546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:30.204129 1546 catalog_manager.cc:1347] Generated new cluster ID: 2f48eacae958413cb989b50dd76aa23b
I20260620 08:13:30.204183 1546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:30.212244 1546 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:30.212419 1546 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:30.220296 1546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61bde287fbfb46f3b2663592aaa8a077: Generated new TSK 0
I20260620 08:13:30.220480 1546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:30.225502 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1459
2026-06-20T08:13:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:30 dist-test-slave-z0sp krb5kdc[1558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:30 dist-test-slave-z0sp krb5kdc[1558](info): set up 2 sockets
Jun 20 08:13:30 dist-test-slave-z0sp krb5kdc[1558](info): commencing operation
krb5kdc: starting...
W20260620 08:13:31.277225 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:31.408252 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:31.491897 1571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:31.492107 1571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:31.492130 1571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:31.493288 1571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:31.493331 1571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:31.493345 1571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:31.493358 1571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:31.493373 1571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:31.494820 1571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:31.495071 1571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:31.495293 1571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:31.500154 1576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:31.500211 1579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:31.500211 1577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:31.500537 1571 server_base.cc:1034] running on GCE node
I20260620 08:13:31.500744 1571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:31.500978 1571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:31.502139 1571 hybrid_clock.cc:648] HybridClock initialized: now 1781943211502125 us; error 26 us; skew 500 ppm
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:31.505131 1571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:31.505635 1571 webserver.cc:460] Webserver started at http://127.0.0.1:36079/ using document root <none> and password file <none>
I20260620 08:13:31.505864 1571 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:31.505934 1571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:31.506085 1571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:31.507112 1571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "0951c3b6e3c048718c752ce2a46bad0b"
format_stamp: "Formatted at 2026-06-20 08:13:31 on dist-test-slave-z0sp"
server_key: "2984b54c8b3d89060ee585a8b5e9029d"
server_key_iv: "60abb4a15819443767ef5d8dc46165f4"
server_key_version: "encryptionkey@0"
I20260620 08:13:31.507474 1571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "0951c3b6e3c048718c752ce2a46bad0b"
format_stamp: "Formatted at 2026-06-20 08:13:31 on dist-test-slave-z0sp"
server_key: "2984b54c8b3d89060ee585a8b5e9029d"
server_key_iv: "60abb4a15819443767ef5d8dc46165f4"
server_key_version: "encryptionkey@0"
I20260620 08:13:31.509057 1571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:31.510210 1586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:31.510460 1571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:31.510584 1571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "0951c3b6e3c048718c752ce2a46bad0b"
format_stamp: "Formatted at 2026-06-20 08:13:31 on dist-test-slave-z0sp"
server_key: "2984b54c8b3d89060ee585a8b5e9029d"
server_key_iv: "60abb4a15819443767ef5d8dc46165f4"
server_key_version: "encryptionkey@0"
I20260620 08:13:31.510695 1571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:31.525132 1571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:31.527125 1571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:31.527284 1571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:31.531345 1571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38037
I20260620 08:13:31.531380 1638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38037 every 8 connection(s)
I20260620 08:13:31.531750 1571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:31.532611 1639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:31.534950 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1571
I20260620 08:13:31.535097 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:31.535271 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: Bootstrap starting.
I20260620 08:13:31.535300 12819 external_mini_cluster.cc:1428] Setting key 03ae9f66a117a32c24cfaf829fc328b7
I20260620 08:13:31.535988 1639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:31.536435 1639 log.cc:824] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:31.537166 1639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: No bootstrap required, opened a new log
I20260620 08:13:31.538952 1639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260620 08:13:31.539108 1639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:31.539155 1639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0951c3b6e3c048718c752ce2a46bad0b, State: Initialized, Role: FOLLOWER
I20260620 08:13:31.539284 1639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260620 08:13:31.539366 1639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:31.539409 1639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:31.539456 1639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:31.540097 1639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
I20260620 08:13:31.540194 1639 leader_election.cc:302] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0951c3b6e3c048718c752ce2a46bad0b; no voters:
I20260620 08:13:31.540426 1639 leader_election.cc:288] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:31.540446 1644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:31.540578 1644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [term 1 LEADER]: Becoming Leader. State: Replica: 0951c3b6e3c048718c752ce2a46bad0b, State: Running, Role: LEADER
I20260620 08:13:31.540729 1639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:31.540741 1644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } }
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 08:13:31.542490 1646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0951c3b6e3c048718c752ce2a46bad0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } } }
I20260620 08:13:31.542586 1645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0951c3b6e3c048718c752ce2a46bad0b. Latest consensus state: current_term: 1 leader_uuid: "0951c3b6e3c048718c752ce2a46bad0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0951c3b6e3c048718c752ce2a46bad0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38037 } } }
I20260620 08:13:31.543036 1646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:31.543045 1645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:31.543397 1659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:31.543447 1659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:31.543509 1661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:31.543669 1661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:31.545205 1661 catalog_manager.cc:1347] Generated new cluster ID: 256250227d914c7f8eeeca5633f55cf9
I20260620 08:13:31.545257 1661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:31.547058 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:31.536163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:32873 (local address 127.0.0.1:38037)
0620 08:13:31.536407 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:31.536418 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:31.536454 (+ 36us) server_negotiation.cc:407] Connection header received
0620 08:13:31.537076 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:31.537083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:31.537131 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:31.537303 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:31.537952 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:31.538823 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:31.539812 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:31.540211 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:31.540329 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:31.543389 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:31.543393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:31.543393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:31.543405 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:31.545496 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:31.545886 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:31.545889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:31.545890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:31.545986 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:31.546215 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:31.546216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:31.546216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:31.546452 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:31.546630 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:31.546751 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 08:13:31.546850 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":116,"threads_started":1}
I20260620 08:13:31.547982 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:31.556169 1661 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:31.556854 1661 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:31.557798 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:31.549819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:39501 (local address 127.0.0.1:38037)
0620 08:13:31.549856 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:31.549858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:31.550106 (+ 248us) server_negotiation.cc:407] Connection header received
0620 08:13:31.550181 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:31.550182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:31.550192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:31.550255 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:31.551134 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:31.551884 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:31.552908 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:31.553218 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:31.553296 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:31.554695 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:31.554697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:31.554697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:31.554704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:31.556482 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:31.556999 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:31.557000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:31.557001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:31.557061 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:31.557296 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:31.557297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:31.557297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:31.557422 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:31.557529 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:31.557638 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 08:13:31.557710 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:13:31.561219 1661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0951c3b6e3c048718c752ce2a46bad0b: Generated new TSK 0
I20260620 08:13:31.561378 1661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:31.565440 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1571
W20260620 08:13:31.569566 1556 connection.cc:535] client connection to 127.0.0.1:38037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38037): Connection reset by peer (error 104)
2026-06-20T08:13:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1670](info): set up 2 sockets
Jun 20 08:13:31 dist-test-slave-z0sp krb5kdc[1670](info): commencing operation
krb5kdc: starting...
W20260620 08:13:32.609102 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:32 dist-test-slave-z0sp krb5kdc[1670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:32.733702 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:32.815222 1683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:32.815447 1683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:32.815482 1683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:32.816617 1683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:32.816685 1683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:32.816713 1683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:32.816754 1683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:32.816778 1683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:32.818207 1683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42111
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:32.818464 1683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:32.818681 1683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:32.823469 1689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:32.823534 1683 server_base.cc:1034] running on GCE node
W20260620 08:13:32.823437 1688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:32.823437 1691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:32.823982 1683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:32.824262 1683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:32.825519 1683 hybrid_clock.cc:648] HybridClock initialized: now 1781943212825482 us; error 41 us; skew 500 ppm
Jun 20 08:13:32 dist-test-slave-z0sp krb5kdc[1670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:32.835433 1683 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:32.836158 1683 webserver.cc:460] Webserver started at http://127.0.0.1:42615/ using document root <none> and password file <none>
I20260620 08:13:32.836404 1683 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:32.836478 1683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:32.836673 1683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:32.837754 1683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "a945ad317dfb4e348571d51933d825a1"
format_stamp: "Formatted at 2026-06-20 08:13:32 on dist-test-slave-z0sp"
server_key: "1a3dcac8c7869e59d69a09246357b36a"
server_key_iv: "1fe704f27299c695728c821544d898a4"
server_key_version: "encryptionkey@0"
I20260620 08:13:32.838133 1683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "a945ad317dfb4e348571d51933d825a1"
format_stamp: "Formatted at 2026-06-20 08:13:32 on dist-test-slave-z0sp"
server_key: "1a3dcac8c7869e59d69a09246357b36a"
server_key_iv: "1fe704f27299c695728c821544d898a4"
server_key_version: "encryptionkey@0"
I20260620 08:13:32.839615 1683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 08:13:32.840581 1698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:32.840821 1683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:32.840909 1683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "a945ad317dfb4e348571d51933d825a1"
format_stamp: "Formatted at 2026-06-20 08:13:32 on dist-test-slave-z0sp"
server_key: "1a3dcac8c7869e59d69a09246357b36a"
server_key_iv: "1fe704f27299c695728c821544d898a4"
server_key_version: "encryptionkey@0"
I20260620 08:13:32.840997 1683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:32.855985 1683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:32.858229 1683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:32.858381 1683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:32.862239 1683 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42111
I20260620 08:13:32.862290 1750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42111 every 8 connection(s)
I20260620 08:13:32.862592 1683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:32.863457 1751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:32.865717 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1: Bootstrap starting.
I20260620 08:13:32.866241 1751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:32.866492 1751 log.cc:824] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:32.867142 1751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1: No bootstrap required, opened a new log
I20260620 08:13:32.868736 1751 raft_consensus.cc:348] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260620 08:13:32.868870 1751 raft_consensus.cc:374] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:32.868891 1751 raft_consensus.cc:729] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a945ad317dfb4e348571d51933d825a1, State: Initialized, Role: FOLLOWER
I20260620 08:13:32.868934 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1683
I20260620 08:13:32.869071 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:32.869076 1751 consensus_queue.cc:260] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260620 08:13:32.869174 1751 raft_consensus.cc:388] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:32.869212 1751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:32.869277 1751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:32.869283 12819 external_mini_cluster.cc:1428] Setting key 3017e0e2edacb473fcb0230e497d9940
I20260620 08:13:32.870429 1751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260620 08:13:32.870534 1751 leader_election.cc:302] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a945ad317dfb4e348571d51933d825a1; no voters:
I20260620 08:13:32.870705 1751 leader_election.cc:288] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:32.870729 1756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:32.870918 1756 raft_consensus.cc:686] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [term 1 LEADER]: Becoming Leader. State: Replica: a945ad317dfb4e348571d51933d825a1, State: Running, Role: LEADER
I20260620 08:13:32.870949 1751 sys_catalog.cc:564] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:32.871062 1756 consensus_queue.cc:237] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260620 08:13:32.872119 1758 sys_catalog.cc:455] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a945ad317dfb4e348571d51933d825a1. Latest consensus state: current_term: 1 leader_uuid: "a945ad317dfb4e348571d51933d825a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260620 08:13:32.872231 1758 sys_catalog.cc:458] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:32.872290 1757 sys_catalog.cc:455] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a945ad317dfb4e348571d51933d825a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a945ad317dfb4e348571d51933d825a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260620 08:13:32.872364 1757 sys_catalog.cc:458] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:32.872535 1770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:32.873121 1770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:32.874490 1770 catalog_manager.cc:1347] Generated new cluster ID: d0743d2583414aefa2c4953769f48837
I20260620 08:13:32.874537 1770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:32.877451 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:32.869817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:38541 (local address 127.0.0.1:42111)
0620 08:13:32.870039 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:32.870051 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:32.870092 (+ 41us) server_negotiation.cc:407] Connection header received
0620 08:13:32.870214 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:32.870221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:32.870277 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:32.870425 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:32.870702 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:32.872177 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:32.873380 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:32.873971 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:32.874113 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:32.876776 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:32.876781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:32.876782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:32.876993 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:32.877029 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:32.877043 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:32.877128 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:32.877222 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 08:13:32.877247 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
I20260620 08:13:32.881505 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:32.888079 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:32.883044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:55975 (local address 127.0.0.1:42111)
0620 08:13:32.883080 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:32.883082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:32.884664 (+ 1582us) server_negotiation.cc:407] Connection header received
0620 08:13:32.884750 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:32.884751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:32.884762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:32.884830 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:32.884993 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:32.885748 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:32.886728 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:32.887082 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:32.887157 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:32.887830 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:32.887831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:32.887831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:32.887837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:32.887851 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:32.887856 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:32.887927 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:32.887990 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 08:13:32.888002 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 08:13:32.894965 1770 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:32.895484 1770 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:32.905654 1770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a945ad317dfb4e348571d51933d825a1: Generated new TSK 0
I20260620 08:13:32.905827 1770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:32.916273 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1683
W20260620 08:13:32.920244 1668 connection.cc:535] client connection to 127.0.0.1:42111 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42111): Connection reset by peer (error 104)
2026-06-20T08:13:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:32 dist-test-slave-z0sp krb5kdc[1782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:32 dist-test-slave-z0sp krb5kdc[1782](info): set up 2 sockets
Jun 20 08:13:32 dist-test-slave-z0sp krb5kdc[1782](info): commencing operation
krb5kdc: starting...
W20260620 08:13:33.969153 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:34 dist-test-slave-z0sp krb5kdc[1782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 08:13:34.097131 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:34.187561 1795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:34.187774 1795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:34.187798 1795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:34.188906 1795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:34.188944 1795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:34.188958 1795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:34.188970 1795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:34.188982 1795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:34.190402 1795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:34.190594 1795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:34.190783 1795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:34.195830 1801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:34.195858 1803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:34.196040 1795 server_base.cc:1034] running on GCE node
W20260620 08:13:34.196103 1800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:34.196357 1795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:34.196723 1795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:34.197923 1795 hybrid_clock.cc:648] HybridClock initialized: now 1781943214197896 us; error 39 us; skew 500 ppm
Jun 20 08:13:34 dist-test-slave-z0sp krb5kdc[1782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:34.200416 1795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 08:13:34.200987 1795 webserver.cc:460] Webserver started at http://127.0.0.1:38497/ using document root <none> and password file <none>
I20260620 08:13:34.201304 1795 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:34.201370 1795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:34.201565 1795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:34.202582 1795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "082062d5e6674720a7fadb05e0fa1e76"
format_stamp: "Formatted at 2026-06-20 08:13:34 on dist-test-slave-z0sp"
server_key: "4da3f2e41cbad01a9a60ac0e5c309423"
server_key_iv: "54f98e9ca1ad0ee1ae43f893eec533d0"
server_key_version: "encryptionkey@0"
I20260620 08:13:34.202955 1795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "082062d5e6674720a7fadb05e0fa1e76"
format_stamp: "Formatted at 2026-06-20 08:13:34 on dist-test-slave-z0sp"
server_key: "4da3f2e41cbad01a9a60ac0e5c309423"
server_key_iv: "54f98e9ca1ad0ee1ae43f893eec533d0"
server_key_version: "encryptionkey@0"
I20260620 08:13:34.204384 1795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 08:13:34.205277 1810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:34.205592 1795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:34.205668 1795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "082062d5e6674720a7fadb05e0fa1e76"
format_stamp: "Formatted at 2026-06-20 08:13:34 on dist-test-slave-z0sp"
server_key: "4da3f2e41cbad01a9a60ac0e5c309423"
server_key_iv: "54f98e9ca1ad0ee1ae43f893eec533d0"
server_key_version: "encryptionkey@0"
I20260620 08:13:34.205717 1795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:34.209050 1795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:34.211257 1795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:34.211366 1795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:34.215430 1795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34595
I20260620 08:13:34.215492 1862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34595 every 8 connection(s)
I20260620 08:13:34.215811 1795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:34.216758 1863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:34.218991 1863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76: Bootstrap starting.
I20260620 08:13:34.219560 1863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:34.219846 1863 log.cc:824] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:34.221151 1863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76: No bootstrap required, opened a new log
I20260620 08:13:34.223408 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1795
I20260620 08:13:34.223551 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:34.223740 12819 external_mini_cluster.cc:1428] Setting key 6789d8ce3690fa30b04a8624761abe09
I20260620 08:13:34.223781 1863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260620 08:13:34.224020 1863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:34.224071 1863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082062d5e6674720a7fadb05e0fa1e76, State: Initialized, Role: FOLLOWER
I20260620 08:13:34.224298 1863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260620 08:13:34.224407 1863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:34.224447 1863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:34.224562 1863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:34.225914 1863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260620 08:13:34.226018 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:34.224310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:56031 (local address 127.0.0.1:34595)
0620 08:13:34.224566 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:34.224578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:34.224603 (+ 25us) server_negotiation.cc:407] Connection header received
0620 08:13:34.224726 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:34.224733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:34.224765 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:34.224892 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:34.225533 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:34.225535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:34.225536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:34.225539 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:34.225578 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:34.225588 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:34.225641 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:34.225734 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 08:13:34.225794 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":135,"threads_started":1}
I20260620 08:13:34.226048 1863 leader_election.cc:302] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 082062d5e6674720a7fadb05e0fa1e76; no voters:
I20260620 08:13:34.226235 1863 leader_election.cc:288] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:34.226282 1868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:34.226444 1868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [term 1 LEADER]: Becoming Leader. State: Replica: 082062d5e6674720a7fadb05e0fa1e76, State: Running, Role: LEADER
I20260620 08:13:34.226613 1868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260620 08:13:34.226486 1863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:34.226972 1870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 082062d5e6674720a7fadb05e0fa1e76. Latest consensus state: current_term: 1 leader_uuid: "082062d5e6674720a7fadb05e0fa1e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } } }
I20260620 08:13:34.227042 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:34.227211 1870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:34.227123 1869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "082062d5e6674720a7fadb05e0fa1e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "082062d5e6674720a7fadb05e0fa1e76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } } }
I20260620 08:13:34.227352 1869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:34.227703 1877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:34.228224 1877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:34.229887 1877 catalog_manager.cc:1347] Generated new cluster ID: 289d666ee8a54e84bfec5fe454e12454
I20260620 08:13:34.229943 1877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:34.234093 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:34.232834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:38497 (local address 127.0.0.1:34595)
0620 08:13:34.232870 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:34.232872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:34.233040 (+ 168us) server_negotiation.cc:407] Connection header received
0620 08:13:34.233104 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:34.233105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:34.233112 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:34.233166 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:34.233845 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:34.233846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:34.233846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:34.233850 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:34.233863 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:34.233868 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:34.233894 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:34.233999 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 08:13:34.234015 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:34.247599 1877 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:34.247781 1877 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:34.257309 1877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 082062d5e6674720a7fadb05e0fa1e76: Generated new TSK 0
I20260620 08:13:34.257496 1877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:34.260106 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1795
2026-06-20T08:13:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:34 dist-test-slave-z0sp krb5kdc[1894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:34 dist-test-slave-z0sp krb5kdc[1894](info): set up 2 sockets
Jun 20 08:13:34 dist-test-slave-z0sp krb5kdc[1894](info): commencing operation
krb5kdc: starting...
W20260620 08:13:35.321312 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:35.469256 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:40593
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:40593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:35.558869 1907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:35.559129 1907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:35.559185 1907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:35.560635 1907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:35.560757 1907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:35.560788 1907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:35.560818 1907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:35.560843 1907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:35.562448 1907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:40593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:40593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:35.562731 1907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:35.562960 1907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:35.568120 1915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:35.568140 1912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:35.568197 1907 server_base.cc:1034] running on GCE node
W20260620 08:13:35.568135 1913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:35.568630 1907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:35.568877 1907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:35.570039 1907 hybrid_clock.cc:648] HybridClock initialized: now 1781943215570023 us; error 32 us; skew 500 ppm
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943215, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:35.573634 1907 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:35.574363 1907 webserver.cc:460] Webserver started at http://127.12.132.254:34995/ using document root <none> and password file <none>
I20260620 08:13:35.574765 1907 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:35.574834 1907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:35.575016 1907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:35.576650 1907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "1ce311cd65354f3685fadfbb3a68fbcd"
format_stamp: "Formatted at 2026-06-20 08:13:35 on dist-test-slave-z0sp"
server_key: "521773f289924e8a774de47f64f31844"
server_key_iv: "fed19a26b0a4741a71829117dc5c42f1"
server_key_version: "encryptionkey@0"
I20260620 08:13:35.577222 1907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "1ce311cd65354f3685fadfbb3a68fbcd"
format_stamp: "Formatted at 2026-06-20 08:13:35 on dist-test-slave-z0sp"
server_key: "521773f289924e8a774de47f64f31844"
server_key_iv: "fed19a26b0a4741a71829117dc5c42f1"
server_key_version: "encryptionkey@0"
I20260620 08:13:35.579312 1907 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260620 08:13:35.580842 1922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:35.581127 1907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:35.581257 1907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "1ce311cd65354f3685fadfbb3a68fbcd"
format_stamp: "Formatted at 2026-06-20 08:13:35 on dist-test-slave-z0sp"
server_key: "521773f289924e8a774de47f64f31844"
server_key_iv: "fed19a26b0a4741a71829117dc5c42f1"
server_key_version: "encryptionkey@0"
I20260620 08:13:35.581369 1907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:35.595844 1907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:35.598513 1907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:35.599052 1907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:35.604533 1907 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:40593
I20260620 08:13:35.604682 1974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:40593 every 8 connection(s)
I20260620 08:13:35.605242 1907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:35.605610 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 1907
I20260620 08:13:35.605788 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:35.606081 12819 external_mini_cluster.cc:1428] Setting key 783d59d8a3b864a05d67ce554ed9326e
I20260620 08:13:35.606740 1975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:35.609688 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd: Bootstrap starting.
I20260620 08:13:35.610311 1975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:35.610605 1975 log.cc:824] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:35.611518 1975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd: No bootstrap required, opened a new log
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:35.614074 1975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } }
I20260620 08:13:35.614279 1975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:35.614319 1975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ce311cd65354f3685fadfbb3a68fbcd, State: Initialized, Role: FOLLOWER
I20260620 08:13:35.614451 1975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } }
I20260620 08:13:35.614557 1975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:35.614599 1975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:35.614650 1975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:35.615222 1975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } }
I20260620 08:13:35.615343 1975 leader_election.cc:302] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ce311cd65354f3685fadfbb3a68fbcd; no voters:
I20260620 08:13:35.615553 1975 leader_election.cc:288] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:35.615584 1980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:35.615777 1980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [term 1 LEADER]: Becoming Leader. State: Replica: 1ce311cd65354f3685fadfbb3a68fbcd, State: Running, Role: LEADER
I20260620 08:13:35.615899 1980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } }
I20260620 08:13:35.615901 1975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:35.616317 1980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce311cd65354f3685fadfbb3a68fbcd. Latest consensus state: current_term: 1 leader_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } } }
I20260620 08:13:35.616415 1980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:35.616640 1985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:35.617084 1981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce311cd65354f3685fadfbb3a68fbcd" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 40593 } } }
I20260620 08:13:35.617165 1985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:35.617589 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:35.606702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59165 (local address 127.12.132.254:40593)
0620 08:13:35.606943 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:35.606955 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:35.606979 (+ 24us) server_negotiation.cc:407] Connection header received
0620 08:13:35.607438 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:35.607444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:35.607489 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:35.607641 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:35.608272 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:35.609265 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:35.610085 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:35.610534 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:35.610684 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:35.613510 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:35.613517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:35.613517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:35.613531 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:35.615534 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:35.616001 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:35.616005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:35.616005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:35.616124 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:35.616322 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:35.616323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:35.616323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:35.616690 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:35.616856 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:35.617277 (+ 421us) server_negotiation.cc:299] Negotiation successful
0620 08:13:35.617396 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":115,"threads_started":1}
I20260620 08:13:35.617969 1981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:35.618698 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:35.619176 1985 catalog_manager.cc:1347] Generated new cluster ID: d2b7ede0fbe540b1b3454182bee45482
I20260620 08:13:35.619233 1985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:35.627388 1978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:35.620184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48371 (local address 127.12.132.254:40593)
0620 08:13:35.620229 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:35.620230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:35.620260 (+ 30us) server_negotiation.cc:407] Connection header received
0620 08:13:35.620314 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:35.620315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:35.620324 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:35.620388 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:35.620972 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:35.621722 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:35.622703 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:35.623031 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:35.623105 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:35.624468 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:35.624470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:35.624470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:35.624477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:35.626078 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:35.626563 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:35.626564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:35.626564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:35.626625 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:35.626875 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:35.626876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:35.626876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:35.627012 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:35.627120 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:35.627235 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 08:13:35.627296 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 08:13:35.646960 1985 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:35.647554 1985 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:35.650513 1985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ce311cd65354f3685fadfbb3a68fbcd: Generated new TSK 0
I20260620 08:13:35.650756 1985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:35.657927 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 1907
W20260620 08:13:35.662940 1892 connection.cc:535] client connection to 127.12.132.254:40593 recv error: Network error: failed to read from TLS socket (remote: 127.12.132.254:40593): Connection reset by peer (error 104)
2026-06-20T08:13:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[2006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[2006](info): set up 2 sockets
Jun 20 08:13:35 dist-test-slave-z0sp krb5kdc[2006](info): commencing operation
krb5kdc: starting...
W20260620 08:13:36.725281 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:36 dist-test-slave-z0sp krb5kdc[2006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:36.856869 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36315
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:44239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:36.939865 2019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:36.940070 2019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:36.940093 2019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:36.941179 2019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:36.941217 2019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:36.941232 2019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:36.941244 2019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:36.941256 2019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:36.942692 2019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:44239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36315
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:36.942924 2019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:36.943092 2019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:36.948308 2024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:36.948370 2027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:36.948341 2025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:36.948741 2019 server_base.cc:1034] running on GCE node
I20260620 08:13:36.949020 2019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:36.949321 2019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:36.950518 2019 hybrid_clock.cc:648] HybridClock initialized: now 1781943216950475 us; error 64 us; skew 500 ppm
Jun 20 08:13:36 dist-test-slave-z0sp krb5kdc[2006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943216, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:36.953228 2019 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:36.953648 2019 webserver.cc:460] Webserver started at http://127.12.132.254:35549/ using document root <none> and password file <none>
I20260620 08:13:36.953889 2019 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:36.953935 2019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:36.954025 2019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:36.955096 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ae90cc0003cc4a9ead60839cec1a9738"
format_stamp: "Formatted at 2026-06-20 08:13:36 on dist-test-slave-z0sp"
server_key: "19e0ade735d35470cd535329407d7b63"
server_key_iv: "e91af8a4a1ce9bbd88a1d22fe82cdc2f"
server_key_version: "encryptionkey@0"
I20260620 08:13:36.955470 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ae90cc0003cc4a9ead60839cec1a9738"
format_stamp: "Formatted at 2026-06-20 08:13:36 on dist-test-slave-z0sp"
server_key: "19e0ade735d35470cd535329407d7b63"
server_key_iv: "e91af8a4a1ce9bbd88a1d22fe82cdc2f"
server_key_version: "encryptionkey@0"
I20260620 08:13:36.957141 2019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:36.958379 2034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:36.958666 2019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:36.958797 2019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ae90cc0003cc4a9ead60839cec1a9738"
format_stamp: "Formatted at 2026-06-20 08:13:36 on dist-test-slave-z0sp"
server_key: "19e0ade735d35470cd535329407d7b63"
server_key_iv: "e91af8a4a1ce9bbd88a1d22fe82cdc2f"
server_key_version: "encryptionkey@0"
I20260620 08:13:36.958891 2019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:36.977824 2019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:36.980463 2019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:36.980666 2019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:36.984614 2019 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:36315
I20260620 08:13:36.984679 2086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:36315 every 8 connection(s)
I20260620 08:13:36.985042 2019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:36.985930 2087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:36.988251 2087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: Bootstrap starting.
I20260620 08:13:36.988860 2087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:36.989161 2087 log.cc:824] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:36.989948 2087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: No bootstrap required, opened a new log
I20260620 08:13:36.991592 2087 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } }
I20260620 08:13:36.991746 2087 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:36.991771 2087 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae90cc0003cc4a9ead60839cec1a9738, State: Initialized, Role: FOLLOWER
I20260620 08:13:36.991911 2087 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } }
I20260620 08:13:36.991988 2087 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:36.991993 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2019
I20260620 08:13:36.992010 2087 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:36.992074 2087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:36.992144 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:36.992566 12819 external_mini_cluster.cc:1428] Setting key 33ca87cd1ff97e5ae77979036a575149
I20260620 08:13:36.993276 2087 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } }
I20260620 08:13:36.993410 2087 leader_election.cc:302] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae90cc0003cc4a9ead60839cec1a9738; no voters:
I20260620 08:13:36.993613 2087 leader_election.cc:288] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:36.993690 2092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:36.993880 2087 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:36.994009 2092 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [term 1 LEADER]: Becoming Leader. State: Replica: ae90cc0003cc4a9ead60839cec1a9738, State: Running, Role: LEADER
I20260620 08:13:36.994204 2092 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } }
I20260620 08:13:36.995012 2093 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae90cc0003cc4a9ead60839cec1a9738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } } }
I20260620 08:13:36.995144 2093 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:36.995484 2094 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae90cc0003cc4a9ead60839cec1a9738. Latest consensus state: current_term: 1 leader_uuid: "ae90cc0003cc4a9ead60839cec1a9738" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae90cc0003cc4a9ead60839cec1a9738" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36315 } } }
I20260620 08:13:36.995822 2094 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738 [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:36.995829 2107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:36.995894 2107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:36.995970 2105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:36.996130 2105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:36.999241 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:36.993346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57337 (local address 127.12.132.254:36315)
0620 08:13:36.993652 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:36.993663 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:36.993700 (+ 37us) server_negotiation.cc:407] Connection header received
0620 08:13:36.993834 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:36.993841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:36.993899 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:36.994312 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:36.994418 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:36.996006 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:36.996925 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:36.997450 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:36.997586 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:36.998507 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:36.998513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:36.998514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:36.998746 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:36.998787 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:36.998803 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:36.998915 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:36.999042 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 08:13:36.999072 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":178,"threads_started":1}
I20260620 08:13:37.000339 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:37.004585 2105 catalog_manager.cc:1347] Generated new cluster ID: 2bc03fc5466c40d2b5b42be41feb0791
I20260620 08:13:37.004642 2105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:37.004982 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:37.001758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57673 (local address 127.12.132.254:36315)
0620 08:13:37.001792 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:37.001793 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:37.001937 (+ 144us) server_negotiation.cc:407] Connection header received
0620 08:13:37.002008 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:37.002009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:37.002033 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:37.002096 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:37.002223 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:37.002961 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:37.003689 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:37.003953 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:37.004029 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:37.004735 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:37.004736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:37.004737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:37.004741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:37.004755 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:37.004759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:37.004812 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:37.004898 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 08:13:37.004910 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260620 08:13:37.013137 2105 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:37.013832 2105 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:37.024807 2105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae90cc0003cc4a9ead60839cec1a9738: Generated new TSK 0
I20260620 08:13:37.024981 2105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:37.031311 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2019
W20260620 08:13:37.036353 2004 connection.cc:535] client connection to 127.12.132.254:36315 recv error: Network error: failed to read from TLS socket (remote: 127.12.132.254:36315): Connection reset by peer (error 104)
2026-06-20T08:13:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:37 dist-test-slave-z0sp krb5kdc[2118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:37 dist-test-slave-z0sp krb5kdc[2118](info): set up 2 sockets
Jun 20 08:13:37 dist-test-slave-z0sp krb5kdc[2118](info): commencing operation
krb5kdc: starting...
W20260620 08:13:38.085202 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:38 dist-test-slave-z0sp krb5kdc[2118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:38.215477 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:34431
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:38.297565 2131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:38.297823 2131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:38.297860 2131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:38.298985 2131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:38.299057 2131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:38.299081 2131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:38.299108 2131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:38.299139 2131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:38.300634 2131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:34431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:38.300908 2131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:38.301131 2131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:38.306596 2139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:38.306634 2137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:38.306645 2131 server_base.cc:1034] running on GCE node
W20260620 08:13:38.306654 2136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:38.307296 2131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:38.307618 2131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:38.308794 2131 hybrid_clock.cc:648] HybridClock initialized: now 1781943218308765 us; error 42 us; skew 500 ppm
Jun 20 08:13:38 dist-test-slave-z0sp krb5kdc[2118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943218, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:38.318275 2131 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:38.318771 2131 webserver.cc:460] Webserver started at http://127.12.132.254:43655/ using document root <none> and password file <none>
I20260620 08:13:38.319020 2131 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:38.319065 2131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:38.319195 2131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:38.320268 2131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "bc451dc7e39743879ba4bcacc1c54d88"
format_stamp: "Formatted at 2026-06-20 08:13:38 on dist-test-slave-z0sp"
server_key: "d35e60ca9e9536b1ee955082d57b814f"
server_key_iv: "a6d43a296deb57f64c1b09d69112867c"
server_key_version: "encryptionkey@0"
I20260620 08:13:38.320647 2131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "bc451dc7e39743879ba4bcacc1c54d88"
format_stamp: "Formatted at 2026-06-20 08:13:38 on dist-test-slave-z0sp"
server_key: "d35e60ca9e9536b1ee955082d57b814f"
server_key_iv: "a6d43a296deb57f64c1b09d69112867c"
server_key_version: "encryptionkey@0"
I20260620 08:13:38.322171 2131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:38.323410 2146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:38.323650 2131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260620 08:13:38.323737 2131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "bc451dc7e39743879ba4bcacc1c54d88"
format_stamp: "Formatted at 2026-06-20 08:13:38 on dist-test-slave-z0sp"
server_key: "d35e60ca9e9536b1ee955082d57b814f"
server_key_iv: "a6d43a296deb57f64c1b09d69112867c"
server_key_version: "encryptionkey@0"
I20260620 08:13:38.323804 2131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:38.326996 2131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:38.330194 2131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:38.330487 2131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:38.335189 2131 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:34431
I20260620 08:13:38.335258 2198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:34431 every 8 connection(s)
I20260620 08:13:38.335618 2131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:38.337016 2199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:38.339354 2199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: Bootstrap starting.
I20260620 08:13:38.339915 2199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:38.340142 2199 log.cc:824] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:38.340883 2199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: No bootstrap required, opened a new log
I20260620 08:13:38.341301 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2131
I20260620 08:13:38.341437 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:38.341640 12819 external_mini_cluster.cc:1428] Setting key f9744ae0b4bf1c9bc4bf7aa8ff51ab65
I20260620 08:13:38.343565 2199 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } }
I20260620 08:13:38.343740 2199 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:38.343779 2199 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc451dc7e39743879ba4bcacc1c54d88, State: Initialized, Role: FOLLOWER
I20260620 08:13:38.343941 2199 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } }
I20260620 08:13:38.344292 2199 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:38.344322 2199 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:38.344383 2199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:38.344929 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:38.342201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49985 (local address 127.12.132.254:34431)
0620 08:13:38.342415 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:38.342424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:38.342457 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:13:38.343517 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:38.343525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:38.343559 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:38.343708 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:38.344457 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:38.344461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:38.344461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:38.344466 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:38.344555 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:38.344569 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:38.344656 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:38.344734 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:13:38.344776 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260620 08:13:38.345103 2199 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } }
I20260620 08:13:38.345204 2199 leader_election.cc:302] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc451dc7e39743879ba4bcacc1c54d88; no voters:
I20260620 08:13:38.345343 2199 leader_election.cc:288] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:38.345418 2204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:38.345598 2199 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:38.345599 2204 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [term 1 LEADER]: Becoming Leader. State: Replica: bc451dc7e39743879ba4bcacc1c54d88, State: Running, Role: LEADER
I20260620 08:13:38.345777 2204 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } }
I20260620 08:13:38.345858 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:38.346131 2204 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc451dc7e39743879ba4bcacc1c54d88. Latest consensus state: current_term: 1 leader_uuid: "bc451dc7e39743879ba4bcacc1c54d88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } } }
I20260620 08:13:38.346256 2204 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:38.347460 2205 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc451dc7e39743879ba4bcacc1c54d88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc451dc7e39743879ba4bcacc1c54d88" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 34431 } } }
I20260620 08:13:38.347566 2205 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88 [sys.catalog]: This master's current role is: LEADER
W20260620 08:13:38.348109 2220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:38.348183 2220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:38.348315 2213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:38.348649 2213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:38.349150 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:38.347849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43473 (local address 127.12.132.254:34431)
0620 08:13:38.347909 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:38.347912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:38.348005 (+ 93us) server_negotiation.cc:407] Connection header received
0620 08:13:38.348118 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:38.348119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:38.348124 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:38.348153 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:38.348897 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:38.348899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:38.348899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:38.348904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:38.348920 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:38.348925 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:38.348996 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:38.349066 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 08:13:38.349080 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 08:13:38.350417 2213 catalog_manager.cc:1347] Generated new cluster ID: 6fecbd0ceef344c3b4e821429e934ed2
I20260620 08:13:38.350471 2213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:38.375392 2213 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:38.375612 2213 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:38.382239 2213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc451dc7e39743879ba4bcacc1c54d88: Generated new TSK 0
I20260620 08:13:38.382611 2213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:38.387642 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2131
2026-06-20T08:13:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:38 dist-test-slave-z0sp krb5kdc[2230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:38 dist-test-slave-z0sp krb5kdc[2230](info): set up 2 sockets
Jun 20 08:13:38 dist-test-slave-z0sp krb5kdc[2230](info): commencing operation
krb5kdc: starting...
W20260620 08:13:39.460989 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:39.597955 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39629
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:39629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:39.685189 2243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:39.685400 2243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:39.685423 2243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:39.686571 2243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:39.686622 2243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:39.686637 2243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:39.686650 2243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:39.686662 2243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:39.688141 2243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:39629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:39629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:39.688387 2243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:39.688643 2243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:39.693986 2249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:39.694031 2248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:39.693979 2251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:39.694401 2243 server_base.cc:1034] running on GCE node
I20260620 08:13:39.694803 2243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:39.695091 2243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:39.696280 2243 hybrid_clock.cc:648] HybridClock initialized: now 1781943219696256 us; error 44 us; skew 500 ppm
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943219, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:39.698913 2243 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:39.699409 2243 webserver.cc:460] Webserver started at http://127.12.132.254:34865/ using document root <none> and password file <none>
I20260620 08:13:39.699637 2243 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:39.699705 2243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:39.699859 2243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:39.701025 2243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "ebcdf1b880a24f1d91115d457ec4406e"
format_stamp: "Formatted at 2026-06-20 08:13:39 on dist-test-slave-z0sp"
server_key: "22e64349a7c51325cdfa3c279bc6aba4"
server_key_iv: "377cfc1866feab8049e5528bd5e9e3cf"
server_key_version: "encryptionkey@0"
I20260620 08:13:39.701398 2243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "ebcdf1b880a24f1d91115d457ec4406e"
format_stamp: "Formatted at 2026-06-20 08:13:39 on dist-test-slave-z0sp"
server_key: "22e64349a7c51325cdfa3c279bc6aba4"
server_key_iv: "377cfc1866feab8049e5528bd5e9e3cf"
server_key_version: "encryptionkey@0"
I20260620 08:13:39.702761 2243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:39.703871 2258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:39.704175 2243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:39.704284 2243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "ebcdf1b880a24f1d91115d457ec4406e"
format_stamp: "Formatted at 2026-06-20 08:13:39 on dist-test-slave-z0sp"
server_key: "22e64349a7c51325cdfa3c279bc6aba4"
server_key_iv: "377cfc1866feab8049e5528bd5e9e3cf"
server_key_version: "encryptionkey@0"
I20260620 08:13:39.704378 2243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:39.717422 2243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:39.719770 2243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:39.720052 2243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:39.725553 2243 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:39629
I20260620 08:13:39.725646 2310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:39629 every 8 connection(s)
I20260620 08:13:39.726089 2243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:39.727032 2311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:39.729324 2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e: Bootstrap starting.
I20260620 08:13:39.729861 2311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:39.730099 2311 log.cc:824] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:39.730839 2311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e: No bootstrap required, opened a new log
I20260620 08:13:39.732424 2311 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } }
I20260620 08:13:39.732650 2311 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:39.732679 2311 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcdf1b880a24f1d91115d457ec4406e, State: Initialized, Role: FOLLOWER
I20260620 08:13:39.732817 2311 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } }
I20260620 08:13:39.732895 2311 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:39.732916 2311 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:39.732944 2311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:39.733731 2311 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } }
I20260620 08:13:39.733824 2311 leader_election.cc:302] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcdf1b880a24f1d91115d457ec4406e; no voters:
I20260620 08:13:39.734040 2311 leader_election.cc:288] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:39.734115 2314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:39.734267 2314 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [term 1 LEADER]: Becoming Leader. State: Replica: ebcdf1b880a24f1d91115d457ec4406e, State: Running, Role: LEADER
I20260620 08:13:39.734342 2311 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:39.734418 2314 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } }
I20260620 08:13:39.734628 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2243
I20260620 08:13:39.734747 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:39.734975 12819 external_mini_cluster.cc:1428] Setting key 08cc69638def390fe7d0160db1ec818e
I20260620 08:13:39.735034 2315 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebcdf1b880a24f1d91115d457ec4406e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } } }
I20260620 08:13:39.735018 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcdf1b880a24f1d91115d457ec4406e. Latest consensus state: current_term: 1 leader_uuid: "ebcdf1b880a24f1d91115d457ec4406e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcdf1b880a24f1d91115d457ec4406e" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 39629 } } }
I20260620 08:13:39.735147 2315 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:39.735152 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:39.735433 2323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:39.736008 2323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:39.737541 2323 catalog_manager.cc:1347] Generated new cluster ID: 350d5212f6264b9791e187ba90458732
I20260620 08:13:39.737594 2323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:39.746937 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:39.735685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33245 (local address 127.12.132.254:39629)
0620 08:13:39.735862 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:39.735872 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:39.735905 (+ 33us) server_negotiation.cc:407] Connection header received
0620 08:13:39.736110 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:39.736118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:39.736249 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:39.736519 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:39.737274 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:39.738639 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:39.739692 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:39.740325 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:39.740447 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:39.743338 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:39.743342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:39.743342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:39.743353 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:39.745302 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:39.745700 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:39.745703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:39.745703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:39.745813 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:39.746021 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:39.746022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:39.746022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:39.746323 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:39.746477 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:39.746591 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 08:13:39.746717 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260620 08:13:39.747829 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:39.756573 2323 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:39.757261 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:39.749239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36707 (local address 127.12.132.254:39629)
0620 08:13:39.749269 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:39.749271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:39.749412 (+ 141us) server_negotiation.cc:407] Connection header received
0620 08:13:39.749454 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:39.749455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:39.749464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:39.749518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:39.750115 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:39.750871 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:39.751913 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:39.752216 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:39.752274 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:39.754471 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:39.754473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:39.754473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:39.754480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:39.756116 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:39.756519 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:39.756520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:39.756520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:39.756575 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:39.756766 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:39.756767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:39.756767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:39.756875 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:39.756980 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:39.757086 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 08:13:39.757163 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260620 08:13:39.757283 2323 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:39.762521 2323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebcdf1b880a24f1d91115d457ec4406e: Generated new TSK 0
I20260620 08:13:39.762724 2323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:39.767804 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2243
2026-06-20T08:13:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2342](info): set up 2 sockets
Jun 20 08:13:39 dist-test-slave-z0sp krb5kdc[2342](info): commencing operation
krb5kdc: starting...
W20260620 08:13:40.833952 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:40 dist-test-slave-z0sp krb5kdc[2342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:40.960362 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45619
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:45619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:41.044952 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:41.045171 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:41.045209 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:41.046370 2355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:41.046542 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:41.046587 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:41.046619 2355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:41.046648 2355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:41.048189 2355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:45619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:45619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:41.048462 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:41.048708 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:41.053821 2363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:41.053860 2361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:41.053898 2355 server_base.cc:1034] running on GCE node
W20260620 08:13:41.053853 2360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:41.054679 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:41.055171 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:41.056358 2355 hybrid_clock.cc:648] HybridClock initialized: now 1781943221056340 us; error 81 us; skew 500 ppm
Jun 20 08:13:41 dist-test-slave-z0sp krb5kdc[2342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943221, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:41.059219 2355 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:41.059659 2355 webserver.cc:460] Webserver started at http://127.12.132.254:39313/ using document root <none> and password file <none>
I20260620 08:13:41.059907 2355 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:41.059978 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:41.060125 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:41.061271 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "16183d5f14834dbba5a5ffa2ee410127"
format_stamp: "Formatted at 2026-06-20 08:13:41 on dist-test-slave-z0sp"
server_key: "ae2d66c10d58211e42ba6a9b990f0bd0"
server_key_iv: "e1ecca99512f8cdce46b9e114e185a02"
server_key_version: "encryptionkey@0"
I20260620 08:13:41.061625 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "16183d5f14834dbba5a5ffa2ee410127"
format_stamp: "Formatted at 2026-06-20 08:13:41 on dist-test-slave-z0sp"
server_key: "ae2d66c10d58211e42ba6a9b990f0bd0"
server_key_iv: "e1ecca99512f8cdce46b9e114e185a02"
server_key_version: "encryptionkey@0"
I20260620 08:13:41.063112 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:41.064282 2370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:41.064625 2355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:41.064772 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "16183d5f14834dbba5a5ffa2ee410127"
format_stamp: "Formatted at 2026-06-20 08:13:41 on dist-test-slave-z0sp"
server_key: "ae2d66c10d58211e42ba6a9b990f0bd0"
server_key_iv: "e1ecca99512f8cdce46b9e114e185a02"
server_key_version: "encryptionkey@0"
I20260620 08:13:41.064875 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:41.091593 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:41.094597 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:41.094786 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:41.100186 2355 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:45619
I20260620 08:13:41.100221 2422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:45619 every 8 connection(s)
I20260620 08:13:41.100781 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:41.101548 2423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:41.104461 2423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127: Bootstrap starting.
I20260620 08:13:41.105129 2423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:41.105437 2423 log.cc:824] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:41.105865 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2355
I20260620 08:13:41.106154 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:41.106392 12819 external_mini_cluster.cc:1428] Setting key 84074ceb27720b34689040b1b32521fa
I20260620 08:13:41.106840 2423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127: No bootstrap required, opened a new log
I20260620 08:13:41.109149 2423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } }
I20260620 08:13:41.109493 2423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:41.109563 2423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16183d5f14834dbba5a5ffa2ee410127, State: Initialized, Role: FOLLOWER
I20260620 08:13:41.109730 2423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } }
I20260620 08:13:41.109810 2423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:41.109877 2423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:41.109934 2423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:41.110702 2423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } }
I20260620 08:13:41.110800 2423 leader_election.cc:302] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16183d5f14834dbba5a5ffa2ee410127; no voters:
I20260620 08:13:41.111033 2423 leader_election.cc:288] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:41.111073 2428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:41.111199 2428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [term 1 LEADER]: Becoming Leader. State: Replica: 16183d5f14834dbba5a5ffa2ee410127, State: Running, Role: LEADER
I20260620 08:13:41.111295 2428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } }
I20260620 08:13:41.111385 2423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:41.111812 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16183d5f14834dbba5a5ffa2ee410127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } } }
I20260620 08:13:41.112202 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:41.112465 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16183d5f14834dbba5a5ffa2ee410127. Latest consensus state: current_term: 1 leader_uuid: "16183d5f14834dbba5a5ffa2ee410127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16183d5f14834dbba5a5ffa2ee410127" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 45619 } } }
I20260620 08:13:41.112646 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:41.112695 2437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:41.113329 2437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:41.115736 2437 catalog_manager.cc:1347] Generated new cluster ID: f25c66b98dce4f87aa6e92ee06639d96
I20260620 08:13:41.115823 2437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:41.116319 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:41.107009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55723 (local address 127.12.132.254:45619)
0620 08:13:41.107247 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:41.107256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:41.107287 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:13:41.109275 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:41.109288 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:41.109371 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:41.109526 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:41.109990 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:41.111952 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:41.113448 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:41.114053 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:41.114222 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:41.115387 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:41.115394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:41.115394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:41.115587 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:41.115652 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:41.115670 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:41.115760 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:41.116050 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 08:13:41.116143 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":105,"threads_started":1}
I20260620 08:13:41.117274 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:41.123466 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:41.118788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40855 (local address 127.12.132.254:45619)
0620 08:13:41.118851 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:41.118852 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:41.120722 (+ 1870us) server_negotiation.cc:407] Connection header received
0620 08:13:41.120823 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:41.120824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:41.120835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:41.120889 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:41.121040 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:41.121520 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:41.122244 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:41.122456 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:41.122535 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:41.123197 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:41.123198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:41.123199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:41.123203 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:41.123223 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:41.123227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:41.123292 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:41.123355 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 08:13:41.123378 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 08:13:41.132601 2437 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:41.133360 2437 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:41.139668 2437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16183d5f14834dbba5a5ffa2ee410127: Generated new TSK 0
I20260620 08:13:41.139842 2437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:41.146313 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2355
2026-06-20T08:13:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:41 dist-test-slave-z0sp krb5kdc[2454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:41 dist-test-slave-z0sp krb5kdc[2454](info): set up 2 sockets
Jun 20 08:13:41 dist-test-slave-z0sp krb5kdc[2454](info): commencing operation
krb5kdc: starting...
W20260620 08:13:42.205855 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:42 dist-test-slave-z0sp krb5kdc[2454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:42.344717 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38345
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:38345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:42.430094 2467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:42.430305 2467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:42.430328 2467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:42.431496 2467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:42.431542 2467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:42.431557 2467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:42.431571 2467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:42.431582 2467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:42.433084 2467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:38345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:38345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:42.433337 2467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:42.433573 2467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:42.438557 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:42.438519 2472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:42.438934 2473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:42.438910 2467 server_base.cc:1034] running on GCE node
I20260620 08:13:42.439157 2467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:42.439414 2467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:42.440659 2467 hybrid_clock.cc:648] HybridClock initialized: now 1781943222440621 us; error 39 us; skew 500 ppm
Jun 20 08:13:42 dist-test-slave-z0sp krb5kdc[2454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943222, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:42.443590 2467 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:42.444033 2467 webserver.cc:460] Webserver started at http://127.12.132.254:36685/ using document root <none> and password file <none>
I20260620 08:13:42.444262 2467 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:42.444316 2467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:42.444481 2467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:42.445559 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "fcb0cf6c171341d1941a21bf20db26c3"
format_stamp: "Formatted at 2026-06-20 08:13:42 on dist-test-slave-z0sp"
server_key: "86cec85a6730e91eb15659cec8c6e632"
server_key_iv: "89dca8de7bacfe9734d5a23c7dd10478"
server_key_version: "encryptionkey@0"
I20260620 08:13:42.445914 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "fcb0cf6c171341d1941a21bf20db26c3"
format_stamp: "Formatted at 2026-06-20 08:13:42 on dist-test-slave-z0sp"
server_key: "86cec85a6730e91eb15659cec8c6e632"
server_key_iv: "89dca8de7bacfe9734d5a23c7dd10478"
server_key_version: "encryptionkey@0"
I20260620 08:13:42.447414 2467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 08:13:42.448380 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:42.448930 2467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:42.449047 2467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "fcb0cf6c171341d1941a21bf20db26c3"
format_stamp: "Formatted at 2026-06-20 08:13:42 on dist-test-slave-z0sp"
server_key: "86cec85a6730e91eb15659cec8c6e632"
server_key_iv: "89dca8de7bacfe9734d5a23c7dd10478"
server_key_version: "encryptionkey@0"
I20260620 08:13:42.449127 2467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:42.452423 2467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:42.455032 2467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:42.455199 2467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:42.459538 2467 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:38345
I20260620 08:13:42.459570 2534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:38345 every 8 connection(s)
I20260620 08:13:42.459934 2467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:42.461098 2535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:42.463258 2535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: Bootstrap starting.
I20260620 08:13:42.463840 2535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:42.464152 2535 log.cc:824] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:42.464882 2535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: No bootstrap required, opened a new log
I20260620 08:13:42.466478 2535 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } }
I20260620 08:13:42.466614 2535 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 08:13:42.466637 2535 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb0cf6c171341d1941a21bf20db26c3, State: Initialized, Role: FOLLOWER
I20260620 08:13:42.466791 2535 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } }
I20260620 08:13:42.466864 2535 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 08:13:42.466886 2535 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 08:13:42.466935 2535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:42.467747 2535 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } }
I20260620 08:13:42.467840 2535 leader_election.cc:302] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb0cf6c171341d1941a21bf20db26c3; no voters:
I20260620 08:13:42.468029 2535 leader_election.cc:288] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 08:13:42.468156 2538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:42.468278 2535 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:42.468345 2538 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [term 1 LEADER]: Becoming Leader. State: Replica: fcb0cf6c171341d1941a21bf20db26c3, State: Running, Role: LEADER
I20260620 08:13:42.468466 2538 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } }
I20260620 08:13:42.468832 2540 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb0cf6c171341d1941a21bf20db26c3. Latest consensus state: current_term: 1 leader_uuid: "fcb0cf6c171341d1941a21bf20db26c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } } }
I20260620 08:13:42.468999 2540 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:42.468827 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb0cf6c171341d1941a21bf20db26c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb0cf6c171341d1941a21bf20db26c3" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 38345 } } }
I20260620 08:13:42.469105 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:42.470099 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2467
I20260620 08:13:42.470232 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:42.470424 12819 external_mini_cluster.cc:1428] Setting key ace4e2704d1ac3349b7c73e4e2eccc18
W20260620 08:13:42.470130 2554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:42.471606 2554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:42.471781 2547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:42.471987 2547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:42.473659 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:42.471083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49937 (local address 127.12.132.254:38345)
0620 08:13:42.471487 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:42.471501 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:42.471644 (+ 143us) server_negotiation.cc:407] Connection header received
0620 08:13:42.471894 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:42.471903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:42.471974 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:42.472189 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:42.473050 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:42.473054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:42.473055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:42.473059 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:42.473100 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:42.473116 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:42.473199 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:42.473403 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 08:13:42.473454 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":98,"threads_started":1}
I20260620 08:13:42.474370 2547 catalog_manager.cc:1347] Generated new cluster ID: 8c436434b6324fa2a9b6fcf42db537e8
I20260620 08:13:42.474424 2547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:42.475098 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:42.482172 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:42.480694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57197 (local address 127.12.132.254:38345)
0620 08:13:42.480784 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:42.480786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:42.480798 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:42.480832 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:42.480833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:42.480841 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:42.480920 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:42.481891 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:42.481892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:42.481893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 08:13:42.481897 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:42.481913 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 08:13:42.481918 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:42.481970 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:42.482082 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 08:13:42.482098 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 08:13:42.482208 2547 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:42.482376 2547 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:42.486802 2547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb0cf6c171341d1941a21bf20db26c3: Generated new TSK 0
I20260620 08:13:42.486994 2547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 08:13:42.495263 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2467
2026-06-20T08:13:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1349 ms)
[----------] 18 tests from AuthTokenIssuingTest (24444 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:42 dist-test-slave-z0sp krb5kdc[2566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:42 dist-test-slave-z0sp krb5kdc[2566](info): set up 2 sockets
Jun 20 08:13:42 dist-test-slave-z0sp krb5kdc[2566](info): commencing operation
krb5kdc: starting...
W20260620 08:13:43.561446 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:43 dist-test-slave-z0sp krb5kdc[2566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:43.701120 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:41227
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:43.783715 2579 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:43.783993 2579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:43.784025 2579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:43.784067 2579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:43.785275 2579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:43.785348 2579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:43.785370 2579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:43.785390 2579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:43.785422 2579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:43.786895 2579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:41227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:43.787171 2579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:43.787395 2579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:43.793361 2584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:43.793555 2579 server_base.cc:1034] running on GCE node
W20260620 08:13:43.793452 2587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:43.793361 2585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:43.793948 2579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:43.794275 2579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:43.795506 2579 hybrid_clock.cc:648] HybridClock initialized: now 1781943223795493 us; error 48 us; skew 500 ppm
Jun 20 08:13:43 dist-test-slave-z0sp krb5kdc[2566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:43.802714 2579 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:43.803400 2579 webserver.cc:460] Webserver started at http://127.12.132.254:37697/ using document root <none> and password file <none>
I20260620 08:13:43.803784 2579 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:43.803882 2579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:43.804060 2579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:43.805307 2579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "2ed58cd346874947adfe53508d1f4a6c"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "17f48865472c2192d1a4b83b0d3a606d"
server_key_iv: "c6463725d7e66fb850353971550a8f51"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.805681 2579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "2ed58cd346874947adfe53508d1f4a6c"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "17f48865472c2192d1a4b83b0d3a606d"
server_key_iv: "c6463725d7e66fb850353971550a8f51"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.807363 2579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:43.808380 2594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:43.808655 2579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:43.808772 2579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "2ed58cd346874947adfe53508d1f4a6c"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "17f48865472c2192d1a4b83b0d3a606d"
server_key_iv: "c6463725d7e66fb850353971550a8f51"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.808866 2579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:43.823026 2579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:43.826285 2579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:43.826700 2579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:43.831331 2579 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:41227
I20260620 08:13:43.831367 2646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:41227 every 8 connection(s)
I20260620 08:13:43.831732 2579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:43.832733 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:43.834297 2647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:43.837143 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2579
I20260620 08:13:43.837265 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:43.837471 12819 external_mini_cluster.cc:1428] Setting key 3ddea24f6d060bb8fb8e921127104a47
Jun 20 08:13:43 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.132.253@KRBTEST.COM; defaulting to no policy
I20260620 08:13:43.847519 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.834968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41227 (local address 127.0.0.1:39844)
0620 08:13:43.835146 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.835199 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:43.835892 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:43.836227 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:43.836230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:43.836614 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:43.836773 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:43.836778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:43.838379 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:43.838380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:43.839422 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:43.839426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:43.839998 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:43.839999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:43.840079 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:43.840734 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:43.840743 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:43.842802 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:43.845576 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:43.845584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:43.845587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:43.845930 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:43.846252 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:43.846253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:43.846253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:43.846362 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:43.846806 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:43.846812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:43.846880 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 08:13:43.847002 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 08:13:43.847280 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":100,"thread_start_us":100,"threads_started":1}
I20260620 08:13:43.847522 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.835251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.12.132.254:41227)
0620 08:13:43.835465 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:43.835470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:43.835510 (+ 40us) server_negotiation.cc:407] Connection header received
0620 08:13:43.836105 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:43.836106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:43.836124 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:43.836223 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:43.836886 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:43.838210 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:43.839545 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:43.839898 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:43.839985 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:43.842978 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:43.842988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:43.842988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:43.842997 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:43.845414 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:43.846061 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:43.846070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:43.846070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:43.846167 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:43.846431 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:43.846432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:43.846432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:43.846721 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:43.846834 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:43.847039 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 08:13:43.847209 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.253@KRBTEST.COM" created.
I20260620 08:13:43.848804 2647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:43.849402 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.849232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59460)
0620 08:13:43.849302 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.849354 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 08:13:43.849627 2597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
I20260620 08:13:43.850167 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.850020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59470)
0620 08:13:43.850111 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.850142 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260620 08:13:43.850327 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:33569: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
I20260620 08:13:43.869678 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:33569
--webserver_interface=127.12.132.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:13:43.899729 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } attempt: 1
I20260620 08:13:43.900427 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.900255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59472)
0620 08:13:43.900338 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.900389 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 08:13:43.900866 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.900766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59488)
0620 08:13:43.900814 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.900846 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 08:13:43.900955 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:33569: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
W20260620 08:13:43.964437 2654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:43.964695 2654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:43.964713 2654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:43.964742 2654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:43.965948 2654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:43.966005 2654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:43.966019 2654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:43.966032 2654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:43.966043 2654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:43.967598 2654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.132.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:43.967873 2654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:43.968101 2654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:13:43.969197 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } attempt: 2
I20260620 08:13:43.969756 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.969593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59500)
0620 08:13:43.969669 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.969718 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 08:13:43.970144 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:43.970044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59506)
0620 08:13:43.970095 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:43.970124 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 08:13:43.970273 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:33569: Network error: Client connection negotiation failed: client connection to 127.12.132.253:33569: connect: Connection refused (error 111)
W20260620 08:13:43.973426 2659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:43.973486 2660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:43.973425 2662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:43.974406 2654 server_base.cc:1034] running on GCE node
I20260620 08:13:43.974720 2654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:43.975008 2654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:43.976275 2654 hybrid_clock.cc:648] HybridClock initialized: now 1781943223976240 us; error 44 us; skew 500 ppm
Jun 20 08:13:43 dist-test-slave-z0sp krb5kdc[2566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:43.979593 2654 init.cc:377] Logged in from keytab as kudu/127.12.132.253@KRBTEST.COM (short username kudu)
I20260620 08:13:43.980211 2654 webserver.cc:460] Webserver started at http://127.12.132.253:36733/ using document root <none> and password file <none>
I20260620 08:13:43.980454 2654 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:43.980554 2654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:43.980732 2654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:43.981870 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data/instance:
uuid: "7d9a10bf72fe4c6a8c5739257aa0139a"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "04d686770aa0e2f5cad27bfc503b9d2b"
server_key_iv: "fab5e0ceceaabbff2473941e5a2bf5f5"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.982679 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal/instance:
uuid: "7d9a10bf72fe4c6a8c5739257aa0139a"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "04d686770aa0e2f5cad27bfc503b9d2b"
server_key_iv: "fab5e0ceceaabbff2473941e5a2bf5f5"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.984561 2654 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260620 08:13:43.985695 2669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:43.986059 2654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:43.986228 2654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "7d9a10bf72fe4c6a8c5739257aa0139a"
format_stamp: "Formatted at 2026-06-20 08:13:43 on dist-test-slave-z0sp"
server_key: "04d686770aa0e2f5cad27bfc503b9d2b"
server_key_iv: "fab5e0ceceaabbff2473941e5a2bf5f5"
server_key_version: "encryptionkey@0"
I20260620 08:13:43.986351 2654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:44.000759 2654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:44.003353 2654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:44.003551 2654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:44.009166 2654 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.253:33569
I20260620 08:13:44.009284 2721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.253:33569 every 8 connection(s)
I20260620 08:13:44.009625 2654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
I20260620 08:13:44.010499 2722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:44.012127 2722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:44.015861 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2654
I20260620 08:13:44.016002 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-1/wal/instance
I20260620 08:13:44.016196 12819 external_mini_cluster.cc:1428] Setting key 2efcac5d208ac8dfe0f851d67a11b701
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:44.022265 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.012917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.12.132.254:41227)
0620 08:13:44.013006 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.013010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.013210 (+ 200us) server_negotiation.cc:407] Connection header received
0620 08:13:44.013781 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.013782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.013797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.013879 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.014514 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.015028 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.016017 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.016311 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.016373 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.018577 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.018579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.018579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.018587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.020292 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.020849 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.020850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.020851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.020924 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.021210 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.021211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.021211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.021396 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.021485 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.022052 (+ 567us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.022146 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 08:13:44.022634 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.012908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41227 (local address 127.0.0.1:39850)
0620 08:13:44.013095 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.013129 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.013636 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.013953 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.013957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.014231 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.014424 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.014427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.015124 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.015125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.015920 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.015922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.016380 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.016381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.016552 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.017110 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.017118 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.018471 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.020437 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.020446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.020449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.020757 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.021035 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.021036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.021036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.021138 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.021546 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.021564 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.021814 (+ 250us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.021999 (+ 185us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.022217 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260620 08:13:44.023488 2722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.132.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.252@KRBTEST.COM" created.
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:44.033258 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.023993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59520)
0620 08:13:44.024044 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.024059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.024120 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.024584 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.024585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.024717 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.024801 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.024803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.025831 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.025832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.026359 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.026361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.026940 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.026941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.027004 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.027666 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.027672 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.029589 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.032009 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.032011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.032011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.032188 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.032457 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.032458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.032458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.032583 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.032989 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.032990 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.033030 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.033087 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.033173 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 08:13:44.033438 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.024053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.12.132.253:33569)
0620 08:13:44.024320 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.024326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.024357 (+ 31us) server_negotiation.cc:407] Connection header received
0620 08:13:44.024384 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.024386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.024401 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.024511 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.024891 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.025696 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.026462 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.026847 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.026908 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.029700 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.029707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.029707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.029714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.031882 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.032288 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.032292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.032292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.032363 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.032653 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.032655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.032655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.032871 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.032970 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.033173 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.033268 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":145,"threads_started":1}
I20260620 08:13:44.034157 2722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:44.034548 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.034427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51736)
0620 08:13:44.034479 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.034527 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 08:13:44.034719 2671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
I20260620 08:13:44.035477 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.035344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51740)
0620 08:13:44.035430 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.035458 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 08:13:44.035724 2722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
I20260620 08:13:44.045833 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:40721
--webserver_interface=127.12.132.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:13:44.061435 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } attempt: 3
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:44.069089 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.061991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59530)
0620 08:13:44.062060 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.062070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.062143 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.062304 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.062305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.062464 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.062532 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.062534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.063205 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.063206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.063721 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.063722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.064135 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.064136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.064217 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.064716 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.064720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.066243 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.068024 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.068025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.068026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.068207 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.068420 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.068421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.068421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.068481 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.068748 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.068750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.068802 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.068882 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.068988 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:13:44.069089 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.062059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.12.132.253:33569)
0620 08:13:44.062129 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.062132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.062142 (+ 10us) server_negotiation.cc:407] Connection header received
0620 08:13:44.062212 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.062213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.062224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.062298 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.062623 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.063115 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.063818 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.064047 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.064133 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.066373 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.066374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.066374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.066381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.067829 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.068288 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.068290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.068290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.068352 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.068556 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.068557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.068558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.068673 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.068745 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.068919 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.068993 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:13:44.069671 2647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:44.070036 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.069940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51742)
0620 08:13:44.069988 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.070019 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 08:13:44.070366 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.070279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51756)
0620 08:13:44.070322 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.070346 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 08:13:44.070459 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
I20260620 08:13:44.085059 2722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 1
I20260620 08:13:44.085830 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.085634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51758)
0620 08:13:44.085713 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.085778 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 08:13:44.086599 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.086469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51770)
0620 08:13:44.086534 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.086575 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 08:13:44.086779 2722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
I20260620 08:13:44.101630 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 1
I20260620 08:13:44.102329 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.102105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51782)
0620 08:13:44.102201 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.102284 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 08:13:44.102849 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.102692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51786)
0620 08:13:44.102756 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.102788 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 08:13:44.102970 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
W20260620 08:13:44.138921 2729 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:44.139146 2729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:44.139163 2729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:44.139184 2729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:44.140285 2729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:44.140324 2729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:44.140338 2729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:44.140352 2729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:44.140363 2729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:44.141834 2729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:41227,127.12.132.253:33569,127.12.132.252:40721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:40721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.132.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:44.142037 2729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:44.142203 2729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:44.148551 2737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:44.148550 2735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:44.148833 2734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:44.148962 2729 server_base.cc:1034] running on GCE node
I20260620 08:13:44.149212 2729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:44.149513 2729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:44.150744 2729 hybrid_clock.cc:648] HybridClock initialized: now 1781943224150683 us; error 38 us; skew 500 ppm
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943224, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:44.153508 2729 init.cc:377] Logged in from keytab as kudu/127.12.132.252@KRBTEST.COM (short username kudu)
I20260620 08:13:44.154026 2729 webserver.cc:460] Webserver started at http://127.12.132.252:38163/ using document root <none> and password file <none>
I20260620 08:13:44.154286 2729 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:44.154371 2729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:44.154523 2729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:44.154958 2722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 2
I20260620 08:13:44.155369 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.155236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51796)
0620 08:13:44.155299 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.155333 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 08:13:44.155694 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.155599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51802)
0620 08:13:44.155645 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.155675 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 08:13:44.155778 2722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
I20260620 08:13:44.155992 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data/instance:
uuid: "5aea33f50a1347928199d60d38daf881"
format_stamp: "Formatted at 2026-06-20 08:13:44 on dist-test-slave-z0sp"
server_key: "da256c99745e864946a4e9ba3abdde17"
server_key_iv: "bb953f8188aee16bc8f6bfff277a7d17"
server_key_version: "encryptionkey@0"
I20260620 08:13:44.156383 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal/instance:
uuid: "5aea33f50a1347928199d60d38daf881"
format_stamp: "Formatted at 2026-06-20 08:13:44 on dist-test-slave-z0sp"
server_key: "da256c99745e864946a4e9ba3abdde17"
server_key_iv: "bb953f8188aee16bc8f6bfff277a7d17"
server_key_version: "encryptionkey@0"
I20260620 08:13:44.157764 2729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 08:13:44.158753 2744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:44.159194 2729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:44.159353 2729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "5aea33f50a1347928199d60d38daf881"
format_stamp: "Formatted at 2026-06-20 08:13:44 on dist-test-slave-z0sp"
server_key: "da256c99745e864946a4e9ba3abdde17"
server_key_iv: "bb953f8188aee16bc8f6bfff277a7d17"
server_key_version: "encryptionkey@0"
I20260620 08:13:44.159440 2729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:44.176651 2729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:44.178194 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 2
I20260620 08:13:44.178648 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.178516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51810)
0620 08:13:44.178581 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.178619 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 08:13:44.179137 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.178909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51822)
0620 08:13:44.179066 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.179106 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260620 08:13:44.179327 2647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:40721: Network error: Client connection negotiation failed: client connection to 127.12.132.252:40721: connect: Connection refused (error 111)
I20260620 08:13:44.179701 2729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:44.179827 2729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:44.184646 2729 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.252:40721
I20260620 08:13:44.184681 2796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.252:40721 every 8 connection(s)
I20260620 08:13:44.185025 2729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
I20260620 08:13:44.185810 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:44.187561 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:44.192876 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2729
I20260620 08:13:44.192984 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781942949664683-12819-0/minicluster-data/master-2/wal/instance
I20260620 08:13:44.193161 12819 external_mini_cluster.cc:1428] Setting key f00f46b35e74ac636c8ec3901097f43d
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943224, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:44.197551 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.188344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.12.132.254:41227)
0620 08:13:44.188469 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.188471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.188671 (+ 200us) server_negotiation.cc:407] Connection header received
0620 08:13:44.189386 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.189387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.189399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.189463 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.190131 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.190602 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.191642 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.191836 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.191916 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.194530 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.194531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.194532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.194539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.196142 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.196591 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.196592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.196593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.196656 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.196916 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.196917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.196917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.197033 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.197103 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.197385 (+ 282us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.197465 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 08:13:44.197734 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.188268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:41227 (local address 127.0.0.1:39866)
0620 08:13:44.188551 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.188585 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.189257 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.189529 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.189532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.189784 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.190032 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.190037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.190759 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.190760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.191538 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.191540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.191922 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.191923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.192066 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.192624 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.192632 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.194442 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.196241 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.196249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.196251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.196471 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.196731 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.196731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.196732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.196841 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.197107 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.197114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.197251 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.197352 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.197496 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":118,"threads_started":1}
I20260620 08:13:44.198308 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943224, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:44.203286 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.193585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.12.132.252:40721)
0620 08:13:44.193811 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.193816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.193842 (+ 26us) server_negotiation.cc:407] Connection header received
0620 08:13:44.193864 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.193866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.193886 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.193966 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.194733 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.195704 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.196527 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.196867 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.196968 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.199797 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.199804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.199804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.199810 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.201997 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.202379 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.202382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.202382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.202443 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.202648 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.202649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.202649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.202815 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.203001 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.203089 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.203173 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":137,"threads_started":1}
I20260620 08:13:44.206327 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.198634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.12.132.253:33569)
0620 08:13:44.198834 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.198836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.198842 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:13:44.198862 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.198863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.198871 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.198924 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.199256 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.199947 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.200959 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.201224 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.201280 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.203070 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.203071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.203072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.203079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.204891 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.205255 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.205256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.205256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.205315 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.205496 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.205497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.205497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.205618 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.205710 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.206195 (+ 485us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.206258 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260620 08:13:44.206352 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.198581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:33569 (local address 127.0.0.1:59534)
0620 08:13:44.198648 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.198661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.198715 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.198957 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.198958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.199096 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.199146 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.199148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.200039 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.200040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.200865 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.200868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.201305 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.201307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.201381 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.201685 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.201691 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.202981 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.204967 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.204968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.204969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.205182 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.205380 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.205381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.205381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.205435 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.205775 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.205777 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.205904 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.206152 (+ 248us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.206263 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 08:13:44.206804 2797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943224, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:44.212929 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.204453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.12.132.254:41227)
0620 08:13:44.204526 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.204527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.204536 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:44.204580 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.204581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.204591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.204657 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.205138 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.205824 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.206888 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.207186 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.207245 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.209610 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.209611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.209612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.209619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.211409 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.211763 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.211764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.211765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.211825 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.212031 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.212032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.212032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.212142 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.212762 (+ 620us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.212796 (+ 34us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.212867 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 08:13:44.214161 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.207070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51830)
0620 08:13:44.207111 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.207120 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.207152 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.207315 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.207316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.207473 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.207514 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.207516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.208186 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.208187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.208848 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.208851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.209320 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.209321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.209391 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.209844 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.209855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.211205 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.212999 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.213000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.213001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.213208 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.213399 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.213400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.213401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.213493 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.213804 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.213805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.213866 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.213943 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.214038 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 08:13:44.214476 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.207185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.12.132.252:40721)
0620 08:13:44.207222 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.207229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.207235 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:13:44.207254 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.207255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.207262 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.207309 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.207614 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.208092 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.208926 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.209231 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.209284 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.211287 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.211288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.211288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.211294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.212911 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.213282 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.213283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.213283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.213340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.213576 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.213577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.213577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.213719 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.213800 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.214115 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.214310 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 08:13:44.216125 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: Bootstrap starting.
I20260620 08:13:44.217257 2797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:44.217551 2797 log.cc:824] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:44.218326 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: No bootstrap required, opened a new log
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:44.219806 2797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.219935 2797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aea33f50a1347928199d60d38daf881, State: Initialized, Role: FOLLOWER
I20260620 08:13:44.220024 2797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.220326 2797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:44.220309 2803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.220534 2803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.221794 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.213666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.12.132.253:33569)
0620 08:13:44.213753 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.213755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.213764 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:44.213787 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.213788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.213798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.213852 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.214441 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.215068 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.215857 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.216278 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.216378 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.218928 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.218930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.218930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.218937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.220631 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.220980 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.220980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.220981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.221063 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.221251 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.221252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.221253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.221398 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.221581 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.221619 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.221693 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260620 08:13:44.222424 2814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:44.222476 2814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:44.223897 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:44.225659 2621 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2ed58cd346874947adfe53508d1f4a6c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39872
I20260620 08:13:44.222517 2729 master_runner.cc:186] Error getting master registration for 127.12.132.254:41227: OK, instance_id { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" instance_seqno: 1781943223830039 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:13:44.227087 2729 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:13:44.246958 2722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 3
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:44.254420 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.247432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.12.132.252:40721)
0620 08:13:44.247504 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.247507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.247515 (+ 8us) server_negotiation.cc:407] Connection header received
0620 08:13:44.247542 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.247543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.247552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.247605 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.247941 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.248419 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.249118 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.249423 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.249542 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.251589 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.251590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.251591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.251597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.253260 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.253586 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.253587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.253588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.253641 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.253851 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.253852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.253852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.254007 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.254086 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.254239 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.254317 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 08:13:44.254420 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.247332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51844)
0620 08:13:44.247394 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.247409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.247462 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.247626 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.247627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.247799 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.247845 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.247847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.248539 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.248540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.249032 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.249034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.249543 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.249544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.249613 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.250084 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.250089 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.251488 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.253338 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.253339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.253340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.253509 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.253718 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.253719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.253720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.253777 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.254078 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.254079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.254125 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.254215 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.254317 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260620 08:13:44.255863 2722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: Bootstrap starting.
I20260620 08:13:44.256459 2722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:44.256759 2722 log.cc:824] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:44.257454 2722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: No bootstrap required, opened a new log
I20260620 08:13:44.258821 2722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.258930 2722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d9a10bf72fe4c6a8c5739257aa0139a, State: Initialized, Role: FOLLOWER
I20260620 08:13:44.258985 2722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.259238 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.259308 2722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:44.259315 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: This master's current role is: FOLLOWER
W20260620 08:13:44.261132 2828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:44.261193 2828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:44.261386 2654 master_runner.cc:186] Error getting master registration for 127.12.132.254:41227: OK, instance_id { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" instance_seqno: 1781943223830039 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:13:44.264259 2654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:13:44.278527 2647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } attempt: 3
Jun 20 08:13:44 dist-test-slave-z0sp krb5kdc[2566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943223, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:44.286180 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.278851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:40721 (local address 127.0.0.1:51850)
0620 08:13:44.278911 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:44.278924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:44.278966 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:44.279148 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:44.279149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:44.279331 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:44.279386 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.279388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.280279 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.280280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.280834 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:44.280836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.281279 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.281280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:44.281343 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.281832 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:44.281841 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:44.283382 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:44.285132 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.285134 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.285134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.285350 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.285548 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:44.285549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:44.285549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:44.285608 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:44.285879 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:44.285880 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:44.285922 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 08:13:44.285984 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 08:13:44.286082 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 08:13:44.286186 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.278918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.12.132.252:40721)
0620 08:13:44.279035 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.279037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.279044 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:13:44.279070 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.279071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.279083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.279132 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.279473 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.280209 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.280920 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.281214 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.281276 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.283454 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.283456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.283456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.283463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.285042 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.285413 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.285414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.285415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.285465 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.285675 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.285676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.285676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.285800 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.285856 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.286038 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.286104 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 08:13:44.288146 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: Bootstrap starting.
I20260620 08:13:44.288913 2647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:44.289207 2647 log.cc:824] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:44.290059 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: No bootstrap required, opened a new log
I20260620 08:13:44.291451 2647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.291587 2647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed58cd346874947adfe53508d1f4a6c, State: Initialized, Role: FOLLOWER
I20260620 08:13:44.291668 2647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.291972 2647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:44.292006 2621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 08:13:44.292048 2621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:44.292910 2621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.293176 2621 leader_election.cc:288] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [CANDIDATE]: Term 1 election: Requested vote from peers 7d9a10bf72fe4c6a8c5739257aa0139a (127.12.132.253:33569), 5aea33f50a1347928199d60d38daf881 (127.12.132.252:40721)
I20260620 08:13:44.293843 2696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ed58cd346874947adfe53508d1f4a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a"
I20260620 08:13:44.293998 2696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:44.294950 2696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed58cd346874947adfe53508d1f4a6c in term 1.
I20260620 08:13:44.296762 2771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ed58cd346874947adfe53508d1f4a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5aea33f50a1347928199d60d38daf881"
I20260620 08:13:44.296892 2771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:44.297798 2771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed58cd346874947adfe53508d1f4a6c in term 1.
W20260620 08:13:44.294184 2842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260620 08:13:44.296060 2597 leader_election.cc:302] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed58cd346874947adfe53508d1f4a6c, 7d9a10bf72fe4c6a8c5739257aa0139a; no voters:
I20260620 08:13:44.300736 2579 master_runner.cc:263] No leader master found from master 2ed58cd346874947adfe53508d1f4a6c
I20260620 08:13:44.300782 2579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 08:13:44.300890 2842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:44.291944 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.301069 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.301153 2831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:44.301237 2831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [term 1 LEADER]: Becoming Leader. State: Replica: 2ed58cd346874947adfe53508d1f4a6c, State: Running, Role: LEADER
I20260620 08:13:44.301371 2831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } }
I20260620 08:13:44.301851 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed58cd346874947adfe53508d1f4a6c. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.301980 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:44.303063 2850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:44.303715 2850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:44.312940 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.302328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.12.132.252:40721)
0620 08:13:44.302369 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.302371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.304651 (+ 2280us) server_negotiation.cc:407] Connection header received
0620 08:13:44.305042 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.305043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.305053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.305082 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.305882 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.306661 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.307731 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.308030 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.308071 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.309720 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.309721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.309721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.309728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.311454 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.311992 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.311994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.311994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.312051 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.312247 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.312248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.312248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.312380 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.312451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.312759 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.312841 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:44.317642 2771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [term 1 FOLLOWER]: Refusing update from remote peer 2ed58cd346874947adfe53508d1f4a6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:44.317966 2848 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:44.319311 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.302933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.12.132.254:41227)
0620 08:13:44.302987 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.302988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.304925 (+ 1937us) server_negotiation.cc:407] Connection header received
0620 08:13:44.304947 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.304948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.304957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.305004 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.311926 (+ 6922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.312717 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.314138 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.314452 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.314517 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.316000 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.316001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.316002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.316009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.317890 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.318480 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.318482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.318482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.318536 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.318734 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.318735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.318735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.318857 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.318940 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.319135 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.319210 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 08:13:44.319391 2803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed58cd346874947adfe53508d1f4a6c. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.319486 2803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.319897 2696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [term 1 FOLLOWER]: Refusing update from remote peer 2ed58cd346874947adfe53508d1f4a6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:44.320137 2848 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:44.320780 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:44.302709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59560 (local address 127.12.132.253:33569)
0620 08:13:44.302751 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.302753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.304745 (+ 1992us) server_negotiation.cc:407] Connection header received
0620 08:13:44.304808 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.304809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.304821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.304891 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.314163 (+ 9272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.314846 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.315821 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:44.316105 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:44.316151 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:44.317559 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:44.317560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:44.317561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:44.317568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:44.319227 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.319695 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.319696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.319696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.319755 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:44.320214 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:44.320215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:44.320216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:44.320367 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:44.320460 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:44.320628 (+ 168us) server_negotiation.cc:299] Negotiation successful
0620 08:13:44.320705 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 08:13:44.321389 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed58cd346874947adfe53508d1f4a6c. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.321477 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.322274 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.322352 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:44.322434 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.322518 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.322685 2854 mvcc.cc:205] Tried to move back new op lower bound from 7298839446801674240 to 7298839446739423232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839446801674240}]
I20260620 08:13:44.329396 2850 catalog_manager.cc:1347] Generated new cluster ID: 28977d0ac9b641a7a12104b3fb293e89
I20260620 08:13:44.329450 2850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:44.329574 2857 mvcc.cc:205] Tried to move back new op lower bound from 7298839446801674240 to 7298839446739423232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839446801674240}]
I20260620 08:13:44.329900 2848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.329986 2848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:44.330291 2803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ed58cd346874947adfe53508d1f4a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed58cd346874947adfe53508d1f4a6c" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 41227 } } peers { permanent_uuid: "7d9a10bf72fe4c6a8c5739257aa0139a" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 33569 } } peers { permanent_uuid: "5aea33f50a1347928199d60d38daf881" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 40721 } } }
I20260620 08:13:44.330363 2803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:44.330931 2855 mvcc.cc:205] Tried to move back new op lower bound from 7298839446801674240 to 7298839446739423232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839446801674240}]
I20260620 08:13:44.344730 2850 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:44.345582 2850 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:44.350498 2850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ed58cd346874947adfe53508d1f4a6c: Generated new TSK 0
I20260620 08:13:44.350730 2850 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:13:44.377776 2801 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.376102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.12.132.252:40721)
0620 08:13:44.376148 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.376150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.377117 (+ 967us) server_negotiation.cc:407] Connection header received
0620 08:13:44.377227 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.377228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.377239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.377280 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.377706 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51874: BlockingRecv error: recv got EOF from 127.0.0.1:51874 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 08:13:44.378516 2650 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.376522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.12.132.254:41227)
0620 08:13:44.376562 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.376564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.378025 (+ 1461us) server_negotiation.cc:407] Connection header received
0620 08:13:44.378118 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.378119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.378129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.378158 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.378457 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39890: BlockingRecv error: recv got EOF from 127.0.0.1:39890 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 08:13:44.379199 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.376848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.12.132.253:33569)
0620 08:13:44.376885 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.376887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.378712 (+ 1825us) server_negotiation.cc:407] Connection header received
0620 08:13:44.378832 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.378833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.378843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.378871 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.379143 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59572: BlockingRecv error: recv got EOF from 127.0.0.1:59572 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 08:13:44.381157 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.380377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.12.132.253:33569)
0620 08:13:44.380410 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.380411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.380692 (+ 281us) server_negotiation.cc:407] Connection header received
0620 08:13:44.380795 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.380796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.380812 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.380840 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.381114 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59578: BlockingRecv error: recv got EOF from 127.0.0.1:59578 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 08:13:44.385547 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.384840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.12.132.253:33569)
0620 08:13:44.384872 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.384873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.385082 (+ 209us) server_negotiation.cc:407] Connection header received
0620 08:13:44.385165 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.385166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.385176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.385202 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.385507 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59590: BlockingRecv error: recv got EOF from 127.0.0.1:59590 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 08:13:44.389430 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.388620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.12.132.253:33569)
0620 08:13:44.388652 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.388653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.388904 (+ 251us) server_negotiation.cc:407] Connection header received
0620 08:13:44.388999 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.389000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.389010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.389038 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.389390 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55754: BlockingRecv error: recv got EOF from 127.0.0.1:55754 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 08:13:44.397622 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.396785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.12.132.253:33569)
0620 08:13:44.396820 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.396822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.397084 (+ 262us) server_negotiation.cc:407] Connection header received
0620 08:13:44.397185 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.397186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.397196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.397225 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.397574 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55764: BlockingRecv error: recv got EOF from 127.0.0.1:55764 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 08:13:44.407718 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.406825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.12.132.253:33569)
0620 08:13:44.406865 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.406867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.407120 (+ 253us) server_negotiation.cc:407] Connection header received
0620 08:13:44.407221 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.407222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.407233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.407276 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.407662 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55766: BlockingRecv error: recv got EOF from 127.0.0.1:55766 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 08:13:44.425772 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.425041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.12.132.253:33569)
0620 08:13:44.425095 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.425098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.425190 (+ 92us) server_negotiation.cc:407] Connection header received
0620 08:13:44.425251 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.425252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.425275 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.425352 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.425715 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55770: BlockingRecv error: recv got EOF from 127.0.0.1:55770 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260620 08:13:44.459798 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.458972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.12.132.253:33569)
0620 08:13:44.459045 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.459047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.459190 (+ 143us) server_negotiation.cc:407] Connection header received
0620 08:13:44.459317 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.459318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.459330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.459409 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.459737 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55784: BlockingRecv error: recv got EOF from 127.0.0.1:55784 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260620 08:13:44.526571 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.525600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.12.132.253:33569)
0620 08:13:44.525745 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.525748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.525895 (+ 147us) server_negotiation.cc:407] Connection header received
0620 08:13:44.525945 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.525946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.525958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.526026 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.526501 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55798: BlockingRecv error: recv got EOF from 127.0.0.1:55798 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260620 08:13:44.657212 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.656031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.12.132.253:33569)
0620 08:13:44.656223 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.656226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.656425 (+ 199us) server_negotiation.cc:407] Connection header received
0620 08:13:44.656481 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.656482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.656538 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.656607 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.657098 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55812: BlockingRecv error: recv got EOF from 127.0.0.1:55812 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260620 08:13:44.915247 2726 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:44.914481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.12.132.253:33569)
0620 08:13:44.914625 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:44.914627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:44.914639 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:44.914674 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:44.914675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:44.914685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:44.914749 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:44.915195 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55818: BlockingRecv error: recv got EOF from 127.0.0.1:55818 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
I20260620 08:13:45.223081 2814 catalog_manager.cc:1259] Loaded cluster ID: 28977d0ac9b641a7a12104b3fb293e89
I20260620 08:13:45.223239 2814 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: loading cluster ID for follower catalog manager: success
I20260620 08:13:45.224171 2814 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: acquiring CA information for follower catalog manager: success
I20260620 08:13:45.224763 2814 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5aea33f50a1347928199d60d38daf881: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:13:45.261816 2828 catalog_manager.cc:1259] Loaded cluster ID: 28977d0ac9b641a7a12104b3fb293e89
I20260620 08:13:45.261930 2828 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: loading cluster ID for follower catalog manager: success
I20260620 08:13:45.263069 2828 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: acquiring CA information for follower catalog manager: success
I20260620 08:13:45.263525 2828 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d9a10bf72fe4c6a8c5739257aa0139a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:13:45.432004 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:45.428950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.12.132.253:33569)
0620 08:13:45.429126 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:45.429129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:45.429234 (+ 105us) server_negotiation.cc:407] Connection header received
0620 08:13:45.429368 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:45.429369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:45.429425 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:45.429517 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:45.429681 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.430204 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.431059 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.431269 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.431361 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:45.431548 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:45.431673 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:45.431759 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:45.431814 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 08:13:45.431837 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
I20260620 08:13:45.437650 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:45.434407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.12.132.252:40721)
0620 08:13:45.434652 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:45.434655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:45.434672 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:13:45.434729 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:45.434730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:45.434782 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:45.434922 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:45.435159 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.435660 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.436465 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.436741 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.436813 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:45.437097 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:45.437270 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:45.437380 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:45.437446 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 08:13:45.437484 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":65,"threads_started":1}
I20260620 08:13:45.443017 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:45.440120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.12.132.253:33569)
0620 08:13:45.440207 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:45.440209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:45.440387 (+ 178us) server_negotiation.cc:407] Connection header received
0620 08:13:45.440535 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:45.440536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:45.440546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:45.440599 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:45.440841 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.441532 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442270 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.442532 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442595 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:45.442742 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:45.442777 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:45.442825 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:45.442902 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 08:13:45.442922 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 08:13:45.443265 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:45.440031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.12.132.252:40721)
0620 08:13:45.440075 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:45.440077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:45.440544 (+ 467us) server_negotiation.cc:407] Connection header received
0620 08:13:45.440606 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:45.440608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:45.440620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:45.440669 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:45.440830 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.441289 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442477 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.442755 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442828 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:45.442985 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:45.443017 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:45.443044 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:45.443158 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 08:13:45.443182 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:13:45.443496 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:45.439997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.12.132.254:41227)
0620 08:13:45.440205 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:45.440207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:45.440216 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:45.440278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:45.440278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:45.440315 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:45.440409 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:45.440559 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.441364 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442451 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:45.442739 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:45.442811 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:45.442994 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:45.443110 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:45.443167 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:45.443395 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 08:13:45.443415 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":55,"threads_started":1}
I20260620 08:13:45.445444 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2579
I20260620 08:13:45.451205 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2654
I20260620 08:13:45.456314 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2729
2026-06-20T08:13:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2962 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 08:13:45 dist-test-slave-z0sp krb5kdc[2945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 08:13:45 dist-test-slave-z0sp krb5kdc[2945](info): set up 2 sockets
Jun 20 08:13:45 dist-test-slave-z0sp krb5kdc[2945](info): commencing operation
krb5kdc: starting...
W20260620 08:13:46.557866 12819 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/test-user.keytab.
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T08:13:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T08:13:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.132.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
Entry for principal kudu/127.12.132.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab.
I20260620 08:13:46.687310 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36035
--webserver_interface=127.12.132.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 08:13:46.769876 2958 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:46.770134 2958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:46.770165 2958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:46.770210 2958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:46.771384 2958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:46.771452 2958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:46.771476 2958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:46.771505 2958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:46.771534 2958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:46.773038 2958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.254.keytab
--principal=kudu/127.12.132.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.254:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.132.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:46.773319 2958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:46.773542 2958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:46.778712 2966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:46.778705 2964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:46.778707 2963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:46.778956 2958 server_base.cc:1034] running on GCE node
I20260620 08:13:46.779209 2958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:46.779469 2958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:46.780659 2958 hybrid_clock.cc:648] HybridClock initialized: now 1781943226780635 us; error 74 us; skew 500 ppm
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:46.783286 2958 init.cc:377] Logged in from keytab as kudu/127.12.132.254@KRBTEST.COM (short username kudu)
I20260620 08:13:46.783730 2958 webserver.cc:460] Webserver started at http://127.12.132.254:34541/ using document root <none> and password file <none>
I20260620 08:13:46.783959 2958 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:46.784029 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:46.784183 2958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:46.785283 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data/instance:
uuid: "bedde35d22df43d7a7d56ead231b8986"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "995815e3277ff48137faa9abb9b9a2d7"
server_key_iv: "996d4a666805a27bc938a912aab2ceb3"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.785633 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance:
uuid: "bedde35d22df43d7a7d56ead231b8986"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "995815e3277ff48137faa9abb9b9a2d7"
server_key_iv: "996d4a666805a27bc938a912aab2ceb3"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.787130 2958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:46.788280 2973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:46.788604 2958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:46.788726 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal
uuid: "bedde35d22df43d7a7d56ead231b8986"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "995815e3277ff48137faa9abb9b9a2d7"
server_key_iv: "996d4a666805a27bc938a912aab2ceb3"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.788820 2958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:46.803491 2958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:46.806679 2958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:46.806896 2958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:46.811722 2958 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.254:36035
I20260620 08:13:46.811803 3025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.254:36035 every 8 connection(s)
I20260620 08:13:46.812136 2958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/data/info.pb
I20260620 08:13:46.812728 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 2958
I20260620 08:13:46.812867 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-0/wal/instance
I20260620 08:13:46.813081 12819 external_mini_cluster.cc:1428] Setting key b3723fc90d55deab1dd08381939388fd
I20260620 08:13:46.813476 3026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:46.816104 3026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.132.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.253@KRBTEST.COM" created.
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:46.830776 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.816986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36035 (local address 127.0.0.1:39004)
0620 08:13:46.817226 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.817277 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:46.817820 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:46.818122 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:46.818125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:46.818391 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:46.818545 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.818548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.820589 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.820591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.821331 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.821334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.822276 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.822277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.822373 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.823004 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:46.823018 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:46.826854 (+ 3836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:46.829240 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.829247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.829250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.829453 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.829707 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.829708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.829708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.829795 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.830168 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:46.830173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:46.830233 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 08:13:46.830341 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 08:13:46.830581 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":92,"thread_start_us":94,"threads_started":1}
I20260620 08:13:46.830785 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.817209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.12.132.254:36035)
0620 08:13:46.817367 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:46.817372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:46.817412 (+ 40us) server_negotiation.cc:407] Connection header received
0620 08:13:46.817982 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:46.817985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:46.818008 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:46.818106 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:46.818649 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.820347 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.821470 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.822126 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.822317 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.826959 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:46.826967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:46.826967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:46.826975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:46.829137 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.829551 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.829554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.829554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.829636 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.829863 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.829863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.829864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.830089 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:46.830179 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:46.830379 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 08:13:46.830516 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260620 08:13:46.831542 3026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:46.832149 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.832016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40334)
0620 08:13:46.832070 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.832116 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 08:13:46.832414 2976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
I20260620 08:13:46.833189 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.833059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40338)
0620 08:13:46.833138 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.833169 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260620 08:13:46.833359 3026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:41027: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
Entry for principal kudu/127.12.132.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab.
I20260620 08:13:46.846125 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:41027
--webserver_interface=127.12.132.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:13:46.882638 3026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } attempt: 1
I20260620 08:13:46.883459 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.883268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40346)
0620 08:13:46.883368 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.883421 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 08:13:46.883939 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.883827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40354)
0620 08:13:46.883889 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.883919 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 08:13:46.884075 3026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:41027: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
W20260620 08:13:46.932893 3033 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:46.933157 3033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:46.933189 3033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:46.933231 3033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:46.934405 3033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:46.934477 3033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:46.934501 3033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:46.934522 3033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:46.934551 3033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:46.936072 3033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.253.keytab
--principal=kudu/127.12.132.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.253:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.132.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:46.936358 3033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:46.936605 3033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 08:13:46.941820 3033 server_base.cc:1034] running on GCE node
W20260620 08:13:46.941761 3039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:46.941756 3041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:46.941756 3038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:46.942286 3033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:46.942585 3033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:46.943783 3033 hybrid_clock.cc:648] HybridClock initialized: now 1781943226943770 us; error 34 us; skew 500 ppm
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:46.946794 3033 init.cc:377] Logged in from keytab as kudu/127.12.132.253@KRBTEST.COM (short username kudu)
I20260620 08:13:46.947248 3033 webserver.cc:460] Webserver started at http://127.12.132.253:37741/ using document root <none> and password file <none>
I20260620 08:13:46.947496 3033 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:46.947566 3033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:46.947722 3033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:46.948908 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data/instance:
uuid: "576612f4a76149a1b3d0102fc78f60e5"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "400618f60b0d853862f18297f2d1751e"
server_key_iv: "71de5ad9ecb564fe3c62cb2a70252b2d"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.949277 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal/instance:
uuid: "576612f4a76149a1b3d0102fc78f60e5"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "400618f60b0d853862f18297f2d1751e"
server_key_iv: "71de5ad9ecb564fe3c62cb2a70252b2d"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.950793 3033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 08:13:46.951956 3048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:46.952226 3033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:46.952227 3026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } attempt: 2
I20260620 08:13:46.952329 3033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal
uuid: "576612f4a76149a1b3d0102fc78f60e5"
format_stamp: "Formatted at 2026-06-20 08:13:46 on dist-test-slave-z0sp"
server_key: "400618f60b0d853862f18297f2d1751e"
server_key_iv: "71de5ad9ecb564fe3c62cb2a70252b2d"
server_key_version: "encryptionkey@0"
I20260620 08:13:46.952415 3033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:46.952656 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.952531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40368)
0620 08:13:46.952583 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.952614 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:13:46.953225 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.953056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40378)
0620 08:13:46.953129 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.953183 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 08:13:46.953562 3026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.253:41027: Network error: Client connection negotiation failed: client connection to 127.12.132.253:41027: connect: Connection refused (error 111)
I20260620 08:13:46.960469 3033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:46.962950 3033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:46.963107 3033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:46.968709 3033 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.253:41027
I20260620 08:13:46.968792 3100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.253:41027 every 8 connection(s)
I20260620 08:13:46.969333 3033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/data/info.pb
I20260620 08:13:46.970435 3101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:46.972023 3101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:46.972666 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 3033
I20260620 08:13:46.972759 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-1/wal/instance
I20260620 08:13:46.972959 12819 external_mini_cluster.cc:1428] Setting key 6a2c32dc2127af1248dba8bdd8fb5f34
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.132.252@KRBTEST.COM; defaulting to no policy
I20260620 08:13:46.983243 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.973186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.12.132.254:36035)
0620 08:13:46.973262 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:46.973264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:46.973529 (+ 265us) server_negotiation.cc:407] Connection header received
0620 08:13:46.974008 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:46.974009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:46.974025 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:46.974107 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:46.974726 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.975409 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.977039 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.977317 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.977387 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.980059 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:46.980061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:46.980062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:46.980072 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:46.981803 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.982253 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.982254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.982255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.982318 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.982578 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.982579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.982579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.982702 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:46.982787 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:46.983064 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 08:13:46.983149 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 08:13:46.983485 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.973178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36035 (local address 127.0.0.1:39016)
0620 08:13:46.973349 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.973414 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:46.973919 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:46.974157 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:46.974160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:46.974414 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:46.974620 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.974624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.975549 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.975551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.976927 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.976930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.977419 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.977420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.977572 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.978154 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:46.978166 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:46.979937 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:46.981910 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.981918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.981921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.982155 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.982387 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.982388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.982388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.982505 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.982771 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:46.982779 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:46.982928 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 08:13:46.983042 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 08:13:46.983203 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.132.252@KRBTEST.COM" created.
I20260620 08:13:46.984562 3101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 08:13:46 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:46.992852 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.984856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40392)
0620 08:13:46.985075 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.985087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:46.985138 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:46.985279 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:46.985280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:46.985422 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:46.985467 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.985469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.986664 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.986665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.987166 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:46.987168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.987765 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.987766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:46.987830 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.988339 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:46.988343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:46.989577 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:46.991737 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.991738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.991738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.991903 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.992113 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:46.992115 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:46.992115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:46.992172 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:46.992469 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:46.992470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:46.992535 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 08:13:46.992625 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 08:13:46.992747 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 08:13:46.993049 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.984935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.12.132.253:41027)
0620 08:13:46.985089 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:46.985094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:46.985137 (+ 43us) server_negotiation.cc:407] Connection header received
0620 08:13:46.985195 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:46.985197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:46.985210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:46.985275 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:46.985553 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.986555 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.987274 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:46.987658 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:46.987765 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:46.989699 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:46.989707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:46.989707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:46.989714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:46.991635 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.991988 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.991991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.991992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.992048 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:46.992234 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:46.992235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:46.992235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:46.992380 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:46.992473 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:46.992668 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 08:13:46.992851 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260620 08:13:46.993664 3101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:46.994012 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.993896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55762)
0620 08:13:46.993946 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.993992 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 08:13:46.994158 3051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
I20260620 08:13:46.994966 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:46.994785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55774)
0620 08:13:46.994855 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:46.994910 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 08:13:46.995139 3101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:46511: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
Entry for principal kudu/127.12.132.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab.
I20260620 08:13:47.003224 12819 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
/tmp/dist-test-taskZ6VfL9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:46511
--webserver_interface=127.12.132.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.132.212:40927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 08:13:47.044404 3101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } attempt: 1
I20260620 08:13:47.044752 3026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } attempt: 3
I20260620 08:13:47.045073 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.044860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55776)
0620 08:13:47.044988 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.045030 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260620 08:13:47.045540 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.045446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55782)
0620 08:13:47.045485 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.045511 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 08:13:47.045643 3101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:46511: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
I20260620 08:13:47.053201 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.045271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.12.132.253:41027)
0620 08:13:47.045310 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.045313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.045403 (+ 90us) server_negotiation.cc:407] Connection header received
0620 08:13:47.045444 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.045445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.045458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.045537 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.046006 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.046505 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.047776 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.048007 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.048079 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.050442 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.050444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.050444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.050452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.051949 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.052366 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.052367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.052367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.052426 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.052678 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.052679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.052679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.052792 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.052865 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.053028 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.053110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 08:13:47.053215 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.045236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40408)
0620 08:13:47.045316 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.045330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.045388 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.045559 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.045560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.045793 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.045905 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.045908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.046630 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.046632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.047659 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.047662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.048135 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.048137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.048256 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.048893 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.048899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.050338 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.052084 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.052086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.052086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.052286 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.052534 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.052534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.052535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.052595 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.052867 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.052868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.052917 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.053003 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.053109 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 08:13:47.053645 3026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:47.054018 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.053905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55796)
0620 08:13:47.053956 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.053989 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 08:13:47.054317 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.054262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55802)
0620 08:13:47.054285 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.054305 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260620 08:13:47.054420 3026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:46511: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
I20260620 08:13:47.085594 3026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } attempt: 1
I20260620 08:13:47.086191 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.086053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55812)
0620 08:13:47.086120 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.086163 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 08:13:47.086886 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.086683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55814)
0620 08:13:47.086784 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.086834 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260620 08:13:47.087203 3026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:46511: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
W20260620 08:13:47.096637 3108 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 08:13:47.096880 3108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 08:13:47.096909 3108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 08:13:47.096971 3108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 08:13:47.098156 3108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 08:13:47.098223 3108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 08:13:47.098264 3108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 08:13:47.098292 3108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 08:13:47.098315 3108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 08:13:47.099825 3108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.132.212:40927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.132.254:36035,127.12.132.253:41027,127.12.132.252:46511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/krb5kdc/kudu_127.12.132.252.keytab
--principal=kudu/127.12.132.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.132.252:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.132.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 06:58:15 UTC on ccycloud.quasar-vppxbf.root.comops.site
build id 4457017
I20260620 08:13:47.100108 3108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 08:13:47.100348 3108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 08:13:47.105279 3113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:47.105284 3114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 08:13:47.105365 3116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 08:13:47.105655 3108 server_base.cc:1034] running on GCE node
I20260620 08:13:47.105899 3108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 08:13:47.106196 3108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 08:13:47.107399 3108 hybrid_clock.cc:648] HybridClock initialized: now 1781943227107385 us; error 30 us; skew 500 ppm
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943227, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 08:13:47.110047 3108 init.cc:377] Logged in from keytab as kudu/127.12.132.252@KRBTEST.COM (short username kudu)
I20260620 08:13:47.110517 3108 webserver.cc:460] Webserver started at http://127.12.132.252:40277/ using document root <none> and password file <none>
I20260620 08:13:47.110780 3108 fs_manager.cc:359] Metadata directory not provided
I20260620 08:13:47.110831 3108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 08:13:47.110957 3108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 08:13:47.112063 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data/instance:
uuid: "b567ae33d84a4856b9994495fe963353"
format_stamp: "Formatted at 2026-06-20 08:13:47 on dist-test-slave-z0sp"
server_key: "4759a5b3d7d6446c7efca1d6915dbf4a"
server_key_iv: "70ee22bf7f29e6b03ea759696d94a117"
server_key_version: "encryptionkey@0"
I20260620 08:13:47.112465 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal/instance:
uuid: "b567ae33d84a4856b9994495fe963353"
format_stamp: "Formatted at 2026-06-20 08:13:47 on dist-test-slave-z0sp"
server_key: "4759a5b3d7d6446c7efca1d6915dbf4a"
server_key_iv: "70ee22bf7f29e6b03ea759696d94a117"
server_key_version: "encryptionkey@0"
I20260620 08:13:47.113829 3101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } attempt: 2
I20260620 08:13:47.113929 3108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:47.114285 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.114139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55828)
0620 08:13:47.114203 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.114243 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 08:13:47.114967 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.114797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55834)
0620 08:13:47.114871 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.114907 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 08:13:47.115068 3101 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.132.252:46511: Network error: Client connection negotiation failed: client connection to 127.12.132.252:46511: connect: Connection refused (error 111)
I20260620 08:13:47.115139 3123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 08:13:47.115300 3108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 08:13:47.115378 3108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data,/tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal
uuid: "b567ae33d84a4856b9994495fe963353"
format_stamp: "Formatted at 2026-06-20 08:13:47 on dist-test-slave-z0sp"
server_key: "4759a5b3d7d6446c7efca1d6915dbf4a"
server_key_iv: "70ee22bf7f29e6b03ea759696d94a117"
server_key_version: "encryptionkey@0"
I20260620 08:13:47.115430 3108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 08:13:47.127539 3108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 08:13:47.129673 3108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 08:13:47.129783 3108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 08:13:47.133936 3108 rpc_server.cc:307] RPC server started. Bound to: 127.12.132.252:46511
I20260620 08:13:47.134097 3175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.132.252:46511 every 8 connection(s)
I20260620 08:13:47.134388 3108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/data/info.pb
I20260620 08:13:47.135432 3176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 08:13:47.137362 3176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:47.140177 12819 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu as pid 3108
I20260620 08:13:47.140276 12819 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZ6VfL9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781942949664683-12819-0/minicluster-data/master-2/wal/instance
I20260620 08:13:47.140645 12819 external_mini_cluster.cc:1428] Setting key 6d738f99fdfc6e4654d68bfcbb779560
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943227, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:47.148217 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.138321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.12.132.254:36035)
0620 08:13:47.138395 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.138401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.138585 (+ 184us) server_negotiation.cc:407] Connection header received
0620 08:13:47.139130 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.139132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.139151 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.139273 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.139940 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.140585 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.141699 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.142018 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.142101 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.144623 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.144624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.144624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.144630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.146177 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.146935 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.146937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.146938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.147112 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.147501 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.147503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.147504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.147638 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.147720 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.148036 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.148122 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 08:13:47.148427 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.138250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.254:36035 (local address 127.0.0.1:39022)
0620 08:13:47.138431 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.138478 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.138946 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.139366 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.139369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.139620 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.139814 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.139818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.140682 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.140683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.141608 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.141610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.142138 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.142139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.142284 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.142913 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.142922 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.144537 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.146328 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.146342 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.146345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.146681 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.147230 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.147231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.147231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.147407 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.147720 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.147728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.147894 (+ 166us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.148014 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.148178 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260620 08:13:47.149161 3176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:47.150453 3180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.141053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.12.132.252:46511)
0620 08:13:47.141210 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.141215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.141264 (+ 49us) server_negotiation.cc:407] Connection header received
0620 08:13:47.141327 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.141330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.141350 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.141421 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.142143 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.143111 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.143973 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.144394 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.144452 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.147203 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.147211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.147211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.147219 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.149013 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.149433 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.149436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.149436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.149497 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.149715 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.149716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.149716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.149885 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.150055 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.150324 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.150386 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943227, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.254@KRBTEST.COM
I20260620 08:13:47.156997 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.149490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.12.132.253:41027)
0620 08:13:47.149564 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.149566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.149578 (+ 12us) server_negotiation.cc:407] Connection header received
0620 08:13:47.149638 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.149639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.149650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.149709 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.150012 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.150901 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.151860 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.152090 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.152144 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.154427 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.154428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.154428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.154435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.155838 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.156218 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.156219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.156219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.156277 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.156458 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.156459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.156459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.156597 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.156674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.156833 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.156905 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 08:13:47.156997 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.149432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.253:41027 (local address 127.0.0.1:40410)
0620 08:13:47.149542 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.149553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.149586 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.149715 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.149716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.149863 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.149899 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.149900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.150998 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.150999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.151761 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.151763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.152161 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.152162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.152231 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.152743 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.152749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.154302 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.155956 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.155957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.155958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.156135 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.156347 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.156348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.156348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.156409 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.156667 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.156668 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.156719 (+ 51us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.156808 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.156903 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260620 08:13:47.157382 3176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } has no permanent_uuid. Determining permanent_uuid...
I20260620 08:13:47.159837 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.151673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.12.132.254:36035)
0620 08:13:47.151711 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.151713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.151722 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:47.151749 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.151750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.151758 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.151835 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.152375 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.153013 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.153850 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.154138 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.154195 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.156800 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.156802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.156802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.156809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.158324 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.158701 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.158702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.158702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.158752 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.158947 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.158947 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.158948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.159071 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.159610 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.159649 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.159729 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943227, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.252@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:47.162380 3026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } attempt: 2
I20260620 08:13:47.164990 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.157654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55838)
0620 08:13:47.157741 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.157761 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.157804 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.157923 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.157924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.158053 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.158078 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.158079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.158939 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.158940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.159713 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.159716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.160299 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.160300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.160351 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.160788 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.160793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.162115 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.163857 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.163858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.163859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.164074 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.164275 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.164277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.164277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.164343 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.164675 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.164676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.164714 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.164784 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.164883 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260620 08:13:47.165038 3180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.157710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55838 (local address 127.12.132.252:46511)
0620 08:13:47.157744 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.157746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.157790 (+ 44us) server_negotiation.cc:407] Connection header received
0620 08:13:47.157858 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.157858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.157868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.157920 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.158148 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.158853 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.159794 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.160200 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.160298 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.162219 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.162220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.162221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.162227 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.163781 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.164139 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.164140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.164140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.164199 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.164447 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.164448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.164448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.164608 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.164684 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.164809 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.164929 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":16}
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.132.253@KRBTEST.COM
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.254@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:47.167513 3176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: Bootstrap starting.
I20260620 08:13:47.168690 3176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:47.168993 3176 log.cc:824] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:47.169965 3176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: No bootstrap required, opened a new log
I20260620 08:13:47.170136 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.160797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.12.132.253:41027)
0620 08:13:47.160942 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.160944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.160962 (+ 18us) server_negotiation.cc:407] Connection header received
0620 08:13:47.160993 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.160994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.161008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.161103 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.161678 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.162271 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.163284 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.163600 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.163647 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.166530 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.166531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.166532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.166539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.168690 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.169062 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.169063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.169063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.169123 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.169490 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.169491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.169491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.169642 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.169819 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.169941 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.170024 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 08:13:47.171228 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.162808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55844)
0620 08:13:47.162857 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.162867 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.162903 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.163307 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.163308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.163457 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.163490 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.163491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.164317 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.164318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.165245 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.165248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.165826 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.165827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.165890 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.166293 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.166298 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.167635 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.170023 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.170024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.170025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.170264 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.170499 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.170500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.170500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.170560 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.170886 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.170887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.170932 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.171017 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.171119 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 08:13:47.171257 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.162870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55844 (local address 127.12.132.252:46511)
0620 08:13:47.163187 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.163189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.163195 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:13:47.163211 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.163212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.163221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.163301 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.163574 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.164221 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.165373 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.165719 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.166029 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.167744 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.167746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.167746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.167753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.169931 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.170355 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.170356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.170357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.170427 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.170636 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.170637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.170638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.170824 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.170894 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.171053 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.171125 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":95,"threads_started":1}
I20260620 08:13:47.171564 12819 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 08:13:47.172232 3176 raft_consensus.cc:348] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.172407 3176 raft_consensus.cc:729] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b567ae33d84a4856b9994495fe963353, State: Initialized, Role: FOLLOWER
I20260620 08:13:47.172545 3176 consensus_queue.cc:260] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.172849 3183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.172904 3176 sys_catalog.cc:564] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:47.173013 3183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.172606 2998 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bedde35d22df43d7a7d56ead231b8986"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39038
I20260620 08:13:47.173884 3026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: Bootstrap starting.
I20260620 08:13:47.174659 3026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:47.174988 3026 log.cc:824] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: Log is configured to *not* fsync() on all Append() calls
W20260620 08:13:47.175880 3194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:47.175941 3194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:47.175055 3108 master_runner.cc:186] Error getting master registration for 127.12.132.254:36035: OK, instance_id { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" instance_seqno: 1781943226810617 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:13:47.177933 3108 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:13:47.178539 3026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: No bootstrap required, opened a new log
I20260620 08:13:47.179557 3026 raft_consensus.cc:348] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.179693 3026 raft_consensus.cc:729] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bedde35d22df43d7a7d56ead231b8986, State: Initialized, Role: FOLLOWER
I20260620 08:13:47.179781 3026 consensus_queue.cc:260] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.180034 3197 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.180087 3026 sys_catalog.cc:564] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:47.180135 2998 raft_consensus.cc:482] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 08:13:47.180215 2998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:47.180142 3197 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.181059 2998 raft_consensus.cc:504] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.181288 2998 leader_election.cc:288] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [CANDIDATE]: Term 1 election: Requested vote from peers 576612f4a76149a1b3d0102fc78f60e5 (127.12.132.253:41027), b567ae33d84a4856b9994495fe963353 (127.12.132.252:46511)
W20260620 08:13:47.182228 3208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:47.182339 3208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:47.182760 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bedde35d22df43d7a7d56ead231b8986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b567ae33d84a4856b9994495fe963353"
I20260620 08:13:47.182871 3150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:47.182967 2958 master_runner.cc:186] Error getting master registration for 127.12.132.253:41027: OK, instance_id { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" instance_seqno: 1781943226966339 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 08:13:47.185045 3150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bedde35d22df43d7a7d56ead231b8986 in term 1.
I20260620 08:13:47.185163 2958 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 08:13:47.185585 2976 leader_election.cc:302] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b567ae33d84a4856b9994495fe963353, bedde35d22df43d7a7d56ead231b8986; no voters:
I20260620 08:13:47.185796 3197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 08:13:47.186100 3197 raft_consensus.cc:686] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [term 1 LEADER]: Becoming Leader. State: Replica: bedde35d22df43d7a7d56ead231b8986, State: Running, Role: LEADER
I20260620 08:13:47.186246 3197 consensus_queue.cc:237] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.187041 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedde35d22df43d7a7d56ead231b8986. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.187248 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:47.187583 3216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 08:13:47.187813 3216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 08:13:47.189553 3150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [term 1 FOLLOWER]: Refusing update from remote peer bedde35d22df43d7a7d56ead231b8986: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:47.189930 3214 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:47.191658 3183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedde35d22df43d7a7d56ead231b8986. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.191761 3183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.201620 3214 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.201735 3214 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:47.202287 3183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.202384 3183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.202659 3219 mvcc.cc:205] Tried to move back new op lower bound from 7298839458565558272 to 7298839458555871232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839458565558272}]
I20260620 08:13:47.202764 3221 mvcc.cc:205] Tried to move back new op lower bound from 7298839458565558272 to 7298839458555871232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839458565558272}]
I20260620 08:13:47.203680 3216 catalog_manager.cc:1347] Generated new cluster ID: dc505fe46f5e4741a2dfc64ebf27e50c
I20260620 08:13:47.203732 3216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 08:13:47.206144 3101 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } attempt: 3
I20260620 08:13:47.206393 3030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.187894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39046 (local address 127.12.132.254:36035)
0620 08:13:47.187936 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.187938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.189052 (+ 1114us) server_negotiation.cc:407] Connection header received
0620 08:13:47.189202 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.189203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.189229 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.189260 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.191384 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.192153 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.196153 (+ 4000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.196584 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.196638 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.201573 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.201575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.201575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.201583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.203603 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.205286 (+ 1683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.205287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.205288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.205354 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.205665 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.205666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.205667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.205807 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.205911 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.206199 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.206284 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 08:13:47 dist-test-slave-z0sp krb5kdc[2945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781943226, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.132.253@KRBTEST.COM for kudu/127.12.132.252@KRBTEST.COM
I20260620 08:13:47.212222 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.188730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.12.132.252:46511)
0620 08:13:47.188770 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.188772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.204634 (+ 15862us) server_negotiation.cc:407] Connection header received
0620 08:13:47.204759 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.204761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.204773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.204805 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.205547 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.206288 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.207315 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.207637 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.207699 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.209078 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.209080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.209080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.209086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.210813 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.211220 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.211221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.211222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.211278 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.211478 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.211478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.211479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.211599 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.211696 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.212025 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.212115 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:13:47.214711 3216 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 08:13:47.215467 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.206610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.132.252:46511 (local address 127.0.0.1:55862)
0620 08:13:47.206695 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 08:13:47.206704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 08:13:47.206740 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 08:13:47.207449 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 08:13:47.207451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 08:13:47.207601 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0620 08:13:47.207647 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.207648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.208570 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.208571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.209379 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 08:13:47.209382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.210005 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.210006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 08:13:47.210077 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.210408 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 08:13:47.210413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 08:13:47.211976 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 08:13:47.214305 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.214306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.214307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.214546 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.214779 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 08:13:47.214780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 08:13:47.214781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 08:13:47.214848 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 08:13:47.215142 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 08:13:47.215143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 08:13:47.215200 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 08:13:47.215269 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 08:13:47.215367 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260620 08:13:47.215467 3216 catalog_manager.cc:1504] Loading token signing keys...
I20260620 08:13:47.215811 3180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.207277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.12.132.252:46511)
0620 08:13:47.207337 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.207339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.207345 (+ 6us) server_negotiation.cc:407] Connection header received
0620 08:13:47.207360 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.207361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.207370 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.207412 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.207737 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.208426 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.209569 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.209885 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.209942 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.212162 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.212163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.212164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.212170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.214132 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.214637 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.214638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.214638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.214702 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.214919 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.214920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.214921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.215049 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.215104 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.215640 (+ 536us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.215725 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 08:13:47.217372 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: Bootstrap starting.
I20260620 08:13:47.218165 3101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: Neither blocks nor log segments found. Creating new log.
I20260620 08:13:47.218479 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:47.188341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.12.132.253:41027)
0620 08:13:47.188392 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.188394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.190136 (+ 1742us) server_negotiation.cc:407] Connection header received
0620 08:13:47.190405 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.190406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.190427 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.190459 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.193654 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.194384 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.198174 (+ 3790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:47.198519 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:47.198561 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:47.202041 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 08:13:47.202043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 08:13:47.202043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 08:13:47.202051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 08:13:47.204374 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.209475 (+ 5101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.209476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.209477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.209540 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 08:13:47.217835 (+ 8295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 08:13:47.217837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 08:13:47.217837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 08:13:47.217973 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 08:13:47.218061 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:47.218291 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 08:13:47.218380 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 08:13:47.218487 3101 log.cc:824] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: Log is configured to *not* fsync() on all Append() calls
I20260620 08:13:47.219972 3101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: No bootstrap required, opened a new log
I20260620 08:13:47.222199 3101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.222350 3101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576612f4a76149a1b3d0102fc78f60e5, State: Initialized, Role: FOLLOWER
I20260620 08:13:47.222431 3101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } }
I20260620 08:13:47.222738 3101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 08:13:47.222865 3074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 08:13:47.223687 3074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [term 1 FOLLOWER]: Refusing update from remote peer bedde35d22df43d7a7d56ead231b8986: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 08:13:47.223984 3214 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [LEADER]: Connected to new peer: Peer: permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 08:13:47.224555 3075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bedde35d22df43d7a7d56ead231b8986" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "576612f4a76149a1b3d0102fc78f60e5"
I20260620 08:13:47.227289 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.227392 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.227447 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedde35d22df43d7a7d56ead231b8986. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.227501 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.227538 3241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.227587 3241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 08:13:47.228463 3216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986: Generated new TSK 0
I20260620 08:13:47.228650 3216 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 08:13:47.228863 3237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 08:13:47.228922 3237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 08:13:47.229271 3075 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bedde35d22df43d7a7d56ead231b8986 for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260620 08:13:47.229699 3197 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bedde35d22df43d7a7d56ead231b8986" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bedde35d22df43d7a7d56ead231b8986" member_type: VOTER last_known_addr { host: "127.12.132.254" port: 36035 } } peers { permanent_uuid: "576612f4a76149a1b3d0102fc78f60e5" member_type: VOTER last_known_addr { host: "127.12.132.253" port: 41027 } } peers { permanent_uuid: "b567ae33d84a4856b9994495fe963353" member_type: VOTER last_known_addr { host: "127.12.132.252" port: 46511 } } }
I20260620 08:13:47.229806 3197 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde35d22df43d7a7d56ead231b8986 [sys.catalog]: This master's current role is: LEADER
I20260620 08:13:47.230867 3239 mvcc.cc:205] Tried to move back new op lower bound from 7298839458665062400 to 7298839458555871232. Current Snapshot: MvccSnapshot[applied={T|T < 7298839458565558272}]
W20260620 08:13:47.253535 3030 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.251870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.12.132.254:36035)
0620 08:13:47.251919 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.251921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.252854 (+ 933us) server_negotiation.cc:407] Connection header received
0620 08:13:47.252963 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.252964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.252975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.253005 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.253451 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39050: BlockingRecv error: recv got EOF from 127.0.0.1:39050 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 08:13:47.254522 3180 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.252574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.12.132.252:46511)
0620 08:13:47.252614 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.252616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.253862 (+ 1246us) server_negotiation.cc:407] Connection header received
0620 08:13:47.254004 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.254005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.254015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.254044 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.254462 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55868: BlockingRecv error: recv got EOF from 127.0.0.1:55868 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 08:13:47.255043 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.252228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.12.132.253:41027)
0620 08:13:47.252292 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.252294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.253736 (+ 1442us) server_negotiation.cc:407] Connection header received
0620 08:13:47.254111 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.254112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.254122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.254152 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.254971 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40450: BlockingRecv error: recv got EOF from 127.0.0.1:40450 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260620 08:13:47.257498 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.256704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40464 (local address 127.12.132.253:41027)
0620 08:13:47.256738 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.256740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.256950 (+ 210us) server_negotiation.cc:407] Connection header received
0620 08:13:47.257054 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.257055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.257073 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.257101 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.257455 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40464: BlockingRecv error: recv got EOF from 127.0.0.1:40464 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 08:13:47.260394 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.259640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.12.132.253:41027)
0620 08:13:47.259671 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.259673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.259910 (+ 237us) server_negotiation.cc:407] Connection header received
0620 08:13:47.260001 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.260002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.260011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.260039 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.260351 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40468: BlockingRecv error: recv got EOF from 127.0.0.1:40468 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260620 08:13:47.264135 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.263415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.12.132.253:41027)
0620 08:13:47.263446 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.263448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.263673 (+ 225us) server_negotiation.cc:407] Connection header received
0620 08:13:47.263767 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.263768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.263777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.263805 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.264093 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40478: BlockingRecv error: recv got EOF from 127.0.0.1:40478 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260620 08:13:47.270077 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.269283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.12.132.253:41027)
0620 08:13:47.269317 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.269319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.269565 (+ 246us) server_negotiation.cc:407] Connection header received
0620 08:13:47.269662 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.269663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.269673 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.269702 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.270028 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40488: BlockingRecv error: recv got EOF from 127.0.0.1:40488 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260620 08:13:47.280104 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.279246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.12.132.253:41027)
0620 08:13:47.279286 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.279288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.279564 (+ 276us) server_negotiation.cc:407] Connection header received
0620 08:13:47.279665 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.279666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.279677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.279705 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.280052 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40498: BlockingRecv error: recv got EOF from 127.0.0.1:40498 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 08:13:47.298509 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.297527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.12.132.253:41027)
0620 08:13:47.297574 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.297577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.297875 (+ 298us) server_negotiation.cc:407] Connection header received
0620 08:13:47.297989 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.297990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.298003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.298034 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.298448 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40508: BlockingRecv error: recv got EOF from 127.0.0.1:40508 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 08:13:47.332860 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.331866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.12.132.253:41027)
0620 08:13:47.331913 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.331914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.332191 (+ 277us) server_negotiation.cc:407] Connection header received
0620 08:13:47.332317 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.332318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.332330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.332361 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.332801 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40514: BlockingRecv error: recv got EOF from 127.0.0.1:40514 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 08:13:47.399145 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.398222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.12.132.253:41027)
0620 08:13:47.398268 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.398271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.398582 (+ 311us) server_negotiation.cc:407] Connection header received
0620 08:13:47.398635 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.398636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.398649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.398710 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.399085 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40516: BlockingRecv error: recv got EOF from 127.0.0.1:40516 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 08:13:47.529376 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.528469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40518 (local address 127.12.132.253:41027)
0620 08:13:47.528583 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.528585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.528716 (+ 131us) server_negotiation.cc:407] Connection header received
0620 08:13:47.528854 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.528855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.528867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.528934 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.529303 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40518: BlockingRecv error: recv got EOF from 127.0.0.1:40518 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260620 08:13:47.788133 3105 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 08:13:47.787081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.12.132.253:41027)
0620 08:13:47.787298 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:47.787300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:47.787309 (+ 9us) server_negotiation.cc:407] Connection header received
0620 08:13:47.787345 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:47.787346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:47.787355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:47.787429 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 08:13:47.788064 (+ 635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40534: BlockingRecv error: recv got EOF from 127.0.0.1:40534 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
I20260620 08:13:48.176704 3194 catalog_manager.cc:1259] Loaded cluster ID: dc505fe46f5e4741a2dfc64ebf27e50c
I20260620 08:13:48.176822 3194 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: loading cluster ID for follower catalog manager: success
I20260620 08:13:48.178038 3194 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: acquiring CA information for follower catalog manager: success
I20260620 08:13:48.178503 3194 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b567ae33d84a4856b9994495fe963353: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:13:48.229370 3237 catalog_manager.cc:1259] Loaded cluster ID: dc505fe46f5e4741a2dfc64ebf27e50c
I20260620 08:13:48.229449 3237 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: loading cluster ID for follower catalog manager: success
I20260620 08:13:48.230216 3237 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: acquiring CA information for follower catalog manager: success
I20260620 08:13:48.230468 3237 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 576612f4a76149a1b3d0102fc78f60e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 08:13:48.305584 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:48.301965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.12.132.253:41027)
0620 08:13:48.302136 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:48.302138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:48.302152 (+ 14us) server_negotiation.cc:407] Connection header received
0620 08:13:48.302206 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:48.302207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:48.302259 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:48.302345 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:48.302663 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.303565 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.304433 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.304655 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.304719 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:48.304980 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:48.305138 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:48.305244 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:48.305375 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 08:13:48.305409 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260620 08:13:48.310490 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:48.307893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55880 (local address 127.12.132.252:46511)
0620 08:13:48.308050 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:48.308052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:48.308059 (+ 7us) server_negotiation.cc:407] Connection header received
0620 08:13:48.308098 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:48.308098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:48.308134 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:48.308224 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:48.308369 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.308909 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.309666 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.309864 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.309967 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:48.310132 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:48.310250 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:48.310335 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:48.310367 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 08:13:48.310387 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260620 08:13:48.315382 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:48.312751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.12.132.252:46511)
0620 08:13:48.312804 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:48.312806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:48.312995 (+ 189us) server_negotiation.cc:407] Connection header received
0620 08:13:48.313054 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:48.313055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:48.313065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:48.313112 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:48.313212 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.313866 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.314661 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.314859 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.314932 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:48.315117 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:48.315151 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:48.315200 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:48.315278 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 08:13:48.315297 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 08:13:48.315557 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:48.312397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.12.132.253:41027)
0620 08:13:48.312440 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:48.312441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:48.312643 (+ 202us) server_negotiation.cc:407] Connection header received
0620 08:13:48.312768 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:48.312769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:48.312781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:48.312838 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:48.312977 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.313565 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.314677 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.314982 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.315022 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:48.315248 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:48.315284 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:48.315333 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:48.315434 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 08:13:48.315461 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 08:13:48.315730 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 08:13:48.312260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.12.132.254:36035)
0620 08:13:48.312391 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 08:13:48.312392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 08:13:48.312409 (+ 17us) server_negotiation.cc:407] Connection header received
0620 08:13:48.312447 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 08:13:48.312447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 08:13:48.312480 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 08:13:48.312628 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 08:13:48.312887 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.313667 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.314670 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 08:13:48.314973 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 08:13:48.315044 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 08:13:48.315227 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 08:13:48.315338 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 08:13:48.315606 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0620 08:13:48.315625 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 08:13:48.315647 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260620 08:13:48.318095 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 2958
I20260620 08:13:48.324608 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 3033
I20260620 08:13:48.331012 12819 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZ6VfL9/build/release/bin/kudu with pid 3108
2026-06-20T08:13:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2875 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5837 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278673 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST